Substitution yielding linear function for least squares fitting

Question

I have an independent variable $\delta$ and dependent set of statistical classified data $\widehat{\Lambda }(\delta)$. The model I want to fit to this data is: $$ \Lambda (\delta )=Ae^{-k\delta}*f(k) $$ There are many cases but for this example we assume: $$ f(k)=\frac{2}{k^2\sigma ^2}\big(e^{\frac{k^2\sigma ^2}{2}\Delta T}-1\big) $$ Note:$\Delta T$ and $\sigma$ are known variables.

In the white paper I've got this equation there was this equation given for the sum of the squared errors, that needs to be minimized: $$ r(A,k)=\sum_{\delta=1}^{\delta_{max}}\bigg(\log\big(\widehat{\Lambda}(\delta)\big)+k\delta-\log\big(A\big)-\log\big(f(k)\big)\bigg)^2 $$ Now since $f(k)$ is not dependent on $\delta$ I would substitute: $$ u=A*f(k) $$ Which gives you this equation as the model: $$ \Lambda(\delta)=ue^{-k\delta} $$ The function of the squared errors that needs to be minimized for this quasilinear model is : $$ r(A,k)=\sum_{\delta=1}^{\delta_{max}}\bigg(\log\big(\widehat{\Lambda}(\delta)\big)+k\delta-\log\big(u\big)\bigg)^2 $$ Now we have a linear equation we need to do our regression for, which is way easier than any non-linear approach. And with $k$ and $\log(u)$ in hand we can easily solve for $A$

I did some back tests with scipy's optimization functions and I got the same results when I optimized the parameters $A$ and $k$ with the initial model and the Levenberg-Marquardt optimization function, and when I tried fit $\log(u)$ and $k$ with the linear least squares fitting approach by hand and then solved for $A$.

But I want to ask you, the experts, if this is really mathematically justified.

Many thanks in advance.

Answer 1

This same logarithmic transformation is discussed in Least squares fit for exponential function.

To summarize, the logarithmic transform distorts the data. The solution does not find the parameters $A$ and $k$ which minimizes the original equation.

However, one can use the method of least squares in the core of the problem. The trick is to fix $k$ and then the problem is linear. Solve for $A$, the compute $r^{2}$. Scan along $k$ to find the minimum $r^{2}$.

The good news is that the least squares problem takes its simplest form: the average. Now, to define the problem. Consider a sequence of $m$ measurements of the form $\left\{ \delta_{j}, \Lambda_{j} \right\}_{j=1}^{m}$. For fixed $k$ we have the linear system $$ \left[ \begin{array}{c} e^{-k \delta_{1}} f(k) \\ \vdots \\ e^{-k \delta_{m}} f(k) \end{array} \right] % \left[ \begin{array}{c} A \end{array} \right] % = % \left[ \begin{array}{c} \Lambda (\delta_{1}) \\ \vdots \\ \Lambda (\delta_{m}) \end{array} \right]. $$ With the restriction $f(k) \ne 0$, a more recognizable form is $$ \left[ \begin{array}{c} 1 \\ \vdots \\ 1 \end{array} \right] % \left[ \begin{array}{c} A \end{array} \right] % = % \left[ \begin{array}{c} \frac{\Lambda (\delta_{1})} {f(k)} e^{k\delta_{1}} \\ \vdots \\ \frac{\Lambda (\delta_{m})} {f(k)} e^{k\delta_{m}} \end{array} \right]. $$ The least squares solution is the average: $$ A(k) = m^{-1} \sum_{j=1}^{m} \frac{\Lambda (\delta_{j})} {f(k)} e^{k\delta_{j}} $$ To connect this with least squares, note this solution is given by the Moore-Penrose pseudoinverse: $$ A(k) = % \left[ \begin{array}{c} 1 \\ \vdots \\ 1 \end{array} \right]^{\dagger} % \left[ \begin{array}{c} \frac{\Lambda (\delta_{1})} {f(k)} e^{k\delta_{1}} \\ \vdots \\ \frac{\Lambda (\delta_{m})} {f(k)} e^{k\delta_{m}} \end{array} \right] % = m^{-1} \left[\begin{array}{ccc} 1 & \cdots & 1 \end{array}\right] \left[ \begin{array}{c} \frac{\Lambda (\delta_{1})} {f(k)} e^{k\delta_{1}} \\ \vdots \\ \frac{\Lambda (\delta_{m})} {f(k)} e^{k\delta_{m}} \end{array} \right]. $$ Now the target of minimization, the sum of the squares of the residuals, is a function of $k$ $$ r^{2}(k) = \Bigg\lVert \left[ \begin{array}{c} e^{-k \delta_{1}} f(k) \\ \vdots \\ e^{-k \delta_{m}} f(k) \end{array} \right] % \left[ \begin{array}{c} A(k) \end{array} \right] - \left[ \begin{array}{c} \Lambda (\delta_{1}) \\ \vdots \\ \Lambda (\delta_{m}) \end{array} \right] \Bigg\rVert_{2}^{2} % = % \sum_{j=1}^{m} \left( e^{-k \delta_{j}} f(k) A(k) - \Lambda(\delta_{j}) \right)^{2}. $$

The least squares solution for your problem is defined as $$ k_{LS} = \left\{ k\in\mathbb{R} \colon r^{2}(k) \text{ is minimized}\right\}. $$