I read the book titled "Seventeen Equations that Changed the World" where it explains how the equation

[A] $x_{t+1}=k \ x_t \cdot (1-x_t)$

where $x_t$ is the population of a certain species at generation $t$;
while $x_{t+1}$ is the population of a certain species at the next generation.

invented by Robert May, was the basis for the chaos theory. This is better known as logistic function[1].

Now, as Wikipedia[1] and other sources state, logistic function is described with another equation, that is

[B] $f(x) = {{L} \over {1+e^{-k(x-x_0)}}}$

a completely different form.

The starting issue was: how equation [A] is the same to [B]?

Then, I googled and found an explaination on Quora[2] that is clear:
[B] comes from a differential version of [A], that is

[C] $y'=k \ y \cdot (L-y)$

Long story short, now the actual question is:

  • how derive [C] from [A]? (or vice versa)
  • under which conditions?

[1] https://en.wikipedia.org/wiki/Logistic_function
[2] https://www.quora.com/How-is-the-logistic-function-derived

  • 3
    $\begingroup$ You don't derive C from A; you note that they are analogous, one being discrete, the other continuous. Each describes a situation where the growth of some quantity (say, a population) is proportional to the current value of the quantity but also to the room left for the quantity to grow when there is some upper bound on how big it can get. $\endgroup$ – Gerry Myerson Aug 20 at 13:11
  • $\begingroup$ Almost the same question: math.stackexchange.com/q/3032887/269624 $\endgroup$ – Yuriy S Aug 20 at 17:29

Subtract $x_t$ to the LHS and RHS of [A] :

$$x_{t+1}-x_t=k x_t (1-x_t)- k \frac{1}{k}x_t $$

$$\underbrace{\dfrac{x_{t+1}-x_t}{1}}_{\text{Discrete derivative}}=k x_t(1 - L x_t) \ \ \text{with} \ \ L:=1+\frac{1}{k}$$

Or, better, under the form (thanks to @Yuriy S for this remark) :

$$\underbrace{\dfrac{x_{t+1}-x_t}{\Delta t}}_{\text{Discrete derivative}}=k' x_t(1 - L x_t) \ \ \text{with} \ \ L:=1+\frac{1}{k}$$

where $\Delta t$ is "small" and $k'$ a new constant, establishing the equivalence between [A] and [C] by assimilation of the discrete and continuous derivatives.

See https://www.zoology.ubc.ca/~bio301/Bio301/Lectures/Lecture5/Overheads.html and the very "didactic" document : http://www.dankalman.net/AUhome/atlanta17JMM/kalman_logisitc_paper.pdf

  • $\begingroup$ Does the "discrete <-> continuous" change come from t: for Robert May, t are generations, thus a discrete variable? $\endgroup$ – mattia.b89 Aug 20 at 12:55
  • $\begingroup$ @mattia.b89 But, "physically", $t$ denotes time in both cases, in the first case considered e.g., as years and in the second case... as usual, continuous time. $\endgroup$ – Jean Marie Aug 20 at 13:08
  • 1
    $\begingroup$ You can make this answer even better by explicitly taking the limit $\Delta t \to 0$, after replacing $1$ by $\Delta t$. It all comes down to scale, since in all applications time has a unit of measure, and one can't usually specify a generation in real life species (like those darn millennials ;)). Otherwise, this answer is good, +1 from me $\endgroup$ – Yuriy S Aug 20 at 13:16
  • $\begingroup$ @Yuriy S You are right. I find this transition between discrete and continuous models rather fascinating ; I have even asked a question some 3 years ago intending to collect different such examples : math.stackexchange.com/q/1774670/305862 I will add this one to my butterfly collection... $\endgroup$ – Jean Marie Aug 20 at 13:50

As to the title of your question, Logistic function: where does it come from?, I can provide an intuition for the logistic function which is the common interpretation from a machine learning perspective. It seems that the underlying question has already been answered above, but I thought this interpretation could help with your intuition about logistic functions in general.

Imagine you would like to model the probability of a random variable, $X$. Denote the corresponding probability density for this as $p(x)\in[0,1]$. Now given some data, we would like to model $p(x)$ in some way.

The classic way to approach modelling in machine learning is via linear regression. Something of the form of $F = \beta G$, where $F$ is a vector of outputs, $\beta$ is a vector of coefficients, and $G$ a matrix of inputs (classically it would be $Y = \beta X$, but since I already used $X$ as a R.V I would like to avoid confusion). An example of a linear regression can look like: $f:\mathbb{R}^3\rightarrow \mathbb{R}: \beta_0 g + \beta_1 g^2 +\beta_2 g^3 \mapsto ( -\infty,\infty)$ i.e. a cubic function.

However we cannot model a probability via regression so far! The image of the cubic function for example was $( -\infty,\infty)$, and the image for $p(x)$ was $[0,1]$.

Consider the first transformation: $\frac{p(x)}{1-p(x)}\in(0,\infty)$.

Now consider taking the logarithm of this: $\log\left(\frac{p(x)}{1-p(x)}\right)\in(-\infty,\infty)$.

Wow, now we can directly us linear regression to model $\log\left(\frac{p(x)}{1-p(x)}\right)$, so that,

\begin{align} \log\left(\frac{p(x)}{1-p(x)}\right) &= \beta G \\ \left(\frac{p(x)}{1-p(x)}\right) &= e^{\beta G} \\ p(x) &= \frac{1}{1+\exp(-\beta G)} \end{align}

Now choose the a linear regression function for $\beta G$ to match your problem, and what you find is that you are in fact modelling a probability for linear regression!

  • $\begingroup$ It does not seem me so "natural" to consider $\frac{p(x)}{1-p(x)}$... $\endgroup$ – Jean Marie Aug 21 at 0:23
  • $\begingroup$ It is natural for linear regression. It wants the image to be unbounded, and this is a simple to have it occur. $\endgroup$ – pche8701 Aug 21 at 2:45
  • $\begingroup$ Perhaps you have some other motivation in mind which could justify p/1-p ? As far as I am aware this is the main intention? $\endgroup$ – pche8701 Aug 21 at 2:47
  • 1
    $\begingroup$ Also, just as an addtional note, the term, $\frac{p(x)}{1-p(x)}$ is commonly known as "odds" in the stats community, so I suppose in this field it is a natural thing to consider. We want to regress over the odds, for odds prediction. $\endgroup$ – pche8701 Aug 22 at 3:53
  • $\begingroup$ This looks a rather satisfying explanation. Thanks. $\endgroup$ – Jean Marie Aug 22 at 4:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.