# least squares using exponential model

I'm trying to fit values from this model $$y(x)=ae^{−bx}+c$$ where a, b and c are 3 different parameters that I want to find with least squares. So using least squares I want to find the value of a, b and c using this formula

$$\mathrm{argmin}\sum_i \left( ae^{−bx_i}+c - y_i \right)^2$$. This leads me to 3 non linear equations after partial derivation of the previous one with respect a, b and c. Surely there are already the general formula for this case but I cannot find them. Can anybody help me?

The problem is difficult because the model is nonlinear $\cdots$ because of parameter $b$. Just suppose you fix it at a given value; then the model becomes linear and you can compute the values of $a(b)$ and $c(b)$ as well as the value of the objective function $\Phi(b)$ you want to minimize.

So, run different values of $b$ until you notice that the function $\Phi(b)$ went through a minimum. At this point, you then have good estimates of the three parameters and you can start a nonlinear procedure. If you do not want, you can polish the solution using Newton-Raphson method for three equations (the derivatives) and the unknowns $(a,b,c)$.

This is a method I commonly use when the model is nonlinear with respect to a small number of parameters (if more than one, generate the value of the function over a grid). This works quite well.

Let me show you the last model for which I used this procedure.

$$P=\rho RT+(B_0RT-A_0-\frac{C_0}{T^2}+\frac{D_0}{T^3}-\frac{E_0}{T^4})\rho^2+(bRT-a-\frac dT)\rho^3+$$ $$\alpha(a+\frac dT)\rho^6+\frac{c \rho^2}{T^2}(1+\gamma \rho^2)e^{-\gamma \rho^2}$$ and the data are $(T_i,\rho_i,P_i)$ ($R$ being the ideal gas constant). The model is linear with respect to all parameters except $\gamma$. It worked very well.

This pocedure has (at least to me) the advantage of using the standard features of linear models. You can change it to a more mathematical one : use $\Phi'_a$ and $\Phi'_c$ to express everything in terms of $b$ and you are just facing the problem of solving $\Phi'_b=0$ for $b$.

Example for illustration

Uisng the data given on page 17 of the book by JJacquelin, I just performed a few calculations for specific values of $b$. The best three points found correspond to $b=1.50$ ($SSQ=0.183835$), $b=1.75$ ($SSQ=0.135040$), $b=2.00$ ($SSQ=0.164380$); for $b=1.75$, $a=0.328$ and $c=0.556$. Starting the nonlinear regression with these values makes the problem converging in three iterations and the final model is $$y=0.337706 +0.542859 e^{1.77478 x}$$ which corresponds to a sum of squares equal to $0.134660$.

Edit

The key problem is to know where to start searching for a reasonable value of parameter $b$. The general model being $$y=ae^{bx}+c$$ let us consider three data points indexed by $i,j,k$; so we can write $$\frac{y_i-y_j}{y_i-y_k}=\frac{e^{bx_i}-e^{bx_j}}{e^{bx_i}-e^{bx_k}}=\frac{1-e^{b(x_j-x_i)}}{1-e^{b(x_k-x_i)}}$$ Now, select $x_k=\frac{x_i+x_j}2$; this leads to $$\frac{y_k-y_j}{y_i-y_k}=e^{b\frac{x_j-x_i}2}$$ then $b$. Even if there is no such $x_k$ in the table, we can approximate it.

For the example from JJacquelin, I took for $x_i$ and $x_j$ the first and last values ($-0.99$ and $0.981$) and I used $y_k=0.911$ corresponding to the $x$ closest to $-0.0045$. This gives $b=1.7077$; reusing the end points gives $a=0.6063$ and $c=0.3062$ which are not bad at all as first guesses.

This is an example of non-linear regression. The softwares solve this kind of problems thanks to some algoritms with iterative methods, starting from guessed values of the parameters.

http://mathworld.wolfram.com/NonlinearLeastSquaresFitting.html

I would mention a straigtforward method (not iterative, no guessed values needed) :

https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales

No need to read the explanations in French. In case of $$y(x)=a+b e^{cx}$$ see page 17, just apply the equations which are are very simple and understandable in any language.

A rough translation is added below :

• the problem is that I don't want to use software. I have to implement the algoirthm that is behind that software! I know that mathlab or R solve that problem in a very relative fast way, but I have to do it on my own May 10, 2015 at 17:41
• You can implement an algorithm by yourself, either with the recursive method (see the link to mathworld, where the theory is explained) or with the straightforward method (see the link to scribd, only a few equations to copy). May 10, 2015 at 17:49
• thanks @JJacquelin , just a last question due to the fact that I do not know french. You said to look at page 17, but is it the 17th page "printed" on the paper or the 17th page out of 85 written up in the menu of the web page? May 10, 2015 at 17:54
• It is the page 17 printed on the paper. Also, see at the end of my first answer : I added a quick translation. Note that this method do not allows a choice of criterium of fitting. If more precision is required with specific criterium, one can use in addition the non-linear regression method and instead of some guessed values to start, use the approximates already computed with the straightforward method. May 10, 2015 at 21:36