3
$\begingroup$

I have data whic would fit to an exponential function with a constant.

$$y=a\cdot\exp(b\cdot t) + c.$$

Now I can solve an exponential without a constant using least square by taking log of y and making the whole equation linear. Is it possible to use least square to solve it with a constant too ( i can't seem to convert the above to linear form, maybe i am missing something here) or do I have to use a non linear fitting function like nlm in R ?

$\endgroup$
  • $\begingroup$ does $t$ take large negative values? is $b$ positive? $\endgroup$ – hHhh Jun 24 '15 at 13:52
  • $\begingroup$ t>=0 and b is -ve for the dataset I have. $\endgroup$ – silencer Jun 24 '15 at 13:57
3
$\begingroup$

The problem is intrinsically nonlinear since you want to minimize $$SSQ(a,b,c)=\sum_{i=1}^N\big(ae^{bt_i}+c-y_i\big)^2$$ and the nonlinear regression will require good (or at least reasonable and consistent) estimates for the three parameters.

But, suppose that you assign a value to $b$; then defining $z_i=e^{bt_i}$ the problem turns to be linear $(y=az+c)$ and a linear regression will give the value of $a,c$ for the given $b$ as well as the sum of the squares. Try a few values of $b$ untill you see a minimum of $SSQ(b)$. For this approximate value of $b$, you had from the linear regression the corresponding $a$ and $c$ and you are ready to go with the nonlinear regression.

Another approach could be : assume a value of $c$ and rewrite the model as $$y-c=a e^{bt}$$ $$\log(y-c)=\alpha + bt$$ which means that defining $w_i=\log(y_i-c)$, the model is just $z=\alpha + bt$ and a linear regression will give $\alpha$ and $b$. From these, recompute $y_i^*=c+e^{ \alpha + bt_i}$ and the corresponding some of squares $SSQ(c)$. Again, trying different values of $c$ will show a minimum and for the best value of $c$, you know the corresponding $b$ and $a=e^{\alpha}$ and you are ready to go with the nonlinear regression.

It is sure that there is one phase with trial and error but it is very fast.

Edit

You can even have an immediate estimate of parameter $b$ if you take three equally spaced points $t_1$, $t_2$, $t_3$ such that $t_2=\frac{t_1+t_3}2$ and, the corresponding $y_i$'s. After simplification $$\frac{ y_3-y_2}{ y_3-y_1}=\frac{1}{1+e^{\frac{1}{2} b (t_1-t_3)}}$$ from which you get the estimate of $b$ $$b=\frac{2}{t_1-t_3} \log \left(\frac{y_1-y_2}{y_2-y_3}\right)$$ from which $$a=\frac{y_1-y_3}{e^{b t_1}-e^{b t_3}}$$ $$c=y_1-ae^{bt_1}$$ Using the data given in page 18 of JJacquelin's book, let us take (approximate values for the $x$'s) the three points $(-1,0.418)$, $(0,0.911)$, $(1,3.544)$. This immediately gives $b\approx 1.675$, $a\approx 0.606$, $c\approx 0.304$ which are extremely close to the end results by JJacquelin in his book for this specific example.

Having these estimates, just run the nonlinear regression.

This was a trick proposed by Yves Daoust here

$\endgroup$
7
$\begingroup$

A direct method of fitting (no guessed initial values required, no iterative process) is shown below. For the theory, see the paper (pp.16-17) : https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales

enter image description here

$\endgroup$
  • $\begingroup$ Fantastic. Unfortunately, I don't read french. Can you point me to the page on which the integral equation is stated that is used to perform the transformation? Would love to learn more. $\endgroup$ – muaddib Jun 24 '15 at 14:17
  • 1
    $\begingroup$ The integral equation is (6) page 16 : $$y-y_1=-ac(x-x_1)+c\int_{x_1}^x y(u)du$$ Presently, there is no available translation for the whole paper. $\endgroup$ – JJacquelin Jun 24 '15 at 15:28
  • $\begingroup$ I am always impressed by your approaches ! $\endgroup$ – Claude Leibovici Jun 25 '15 at 13:34
  • $\begingroup$ Here's a numpy implementation of the above. Behaves a bit weird sometimes; no guarantees that it is accurate. hastebin.com/yuxoxotipo.py $\endgroup$ – Mingwei Samuel Jul 24 '17 at 22:56
  • 1
    $\begingroup$ @JJacquelin This is very nice. I've started using the approach you provide in your paper(s) in a few places in my work. I've opened up github.com/scipy/scipy/pull/9158. I hope the content (and citation) is to your liking. $\endgroup$ – Mad Physicist Aug 20 '18 at 19:24
1
$\begingroup$

You can solve the problem by regressing over the derivate (difference) of the data. If you formulate the problem as having to solve for $a, b, c$ in

$$y=b\cdot e^{ax} + c$$

By taking the derivative you get:

\begin{align} \frac{dy}{dx} &= ab\cdot e^{ax} \\ \log\left(\frac{dy}{dx}\right) &= \log(ab\cdot e^{ax}) \\ &= \log(ab) + \log(e^{ax}) \\ &= \log(ab) + ax \end{align}

You can then fit a linear model of the form $u = s x + t$ where $u=\log\left(\frac{dy}{dx}\right)$, $s=a$, and $t=\log(ab)$. After solving you can obtain $a = s$ and $b = \frac{e^t}{a}$. And finally, you can obtain c by subtracting the reconstructed model from the original data.

Note: this whole exercise will require you to sort your data set by $x$ values.

Here is python code to accomplish the task:

def regress_exponential_with_offset(x, y):
    # sort values
    ind = np.argsort(x)
    x = x[ind]
    y = y[ind]

    # decaying exponentials need special treatment
    # since we can't take the log of negative numbers.
    neg = -1 if y[0] > y[-1] else 1
    dx = np.diff(x)
    dy = np.diff(y)
    dy_dx = dy / dx

    # filter any remaining negative numbers.
    v = x[:-1]
    u = neg * dy_dx
    ind = np.where(u > 0)[0]
    v = v[ind]
    u = u[ind]

    # perform regression
    u = np.log(u)
    s, t = np.polyfit(v, u, 1)
    a = s
    b = neg * np.exp(t) / a
    yy = np.exp(a * x) * b
    c = np.median(y - yy)
    return a, b, c
$\endgroup$
  • 1
    $\begingroup$ I am sorry but this is extremely dangerous to do (except if the data are strictly perfect). $\endgroup$ – Claude Leibovici Jan 15 at 10:48
  • $\begingroup$ @ClaudeLeibovici Can you please elaborate? I've used this technique successfully on pretty noisy data. $\endgroup$ – gilsho Jan 16 at 13:53
  • $\begingroup$ I did not want to answer as soon as I saw your comment and, thinking, I shall not answer NOW but I am ready to have as long discussions as required on this topic with you (chat room, e-mails, ...) if you accept to first ask the question "Is this correct or not ? at Cross Validated Stack Exchange. Cheers. $\endgroup$ – Claude Leibovici Jan 17 at 2:15
  • 1
    $\begingroup$ @glisho. The idea to use the derivative of a non linear function to transform a non linear into a linear regression is not new. This is a good idea but with important drawbacks in practice, due to hazard in numerical calculus . Such method was described in the paper referenced below. But this is an very uncertain method if the data are scattered and/or noisy as shown in this paper. I fully agree with the comment of Claude Leibovici. In fact it is much better to use antiderivative than derivative because the numerical integration is much accurate and stable than numerical differentiation. $\endgroup$ – JJacquelin Jan 17 at 5:30
  • $\begingroup$ For example, the use of integral equation (but not differential equation) in case of $y=a \exp(bx)+c\quad$ is described pages 15-18 in the paper : fr.scribd.com/doc/14674814/Regressions-et-equations-integrales $\endgroup$ – JJacquelin Jan 17 at 6:56
-1
$\begingroup$

I tried to deduce the formula for 'b' in case of equdistant ti-s, but I have got a different one. There is a reciprocal under the logarithm in my result. (Please check it. Maybe I am not right.)

Sorry. I see ... The t3-t1 minus sign makes the log content right.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.