2
$\begingroup$

I am currently studying at college curve fitting to a set of experimental data. One of our activities/homework was to fit the curve $y=ax^b$ to the set of points of the position of a free-falling object we filmed. My professor stated that it was always necessary to first linearize the data, by taking the logarithm of the $x$ and $y$ values, and then do a least squares regression on this new data, and that the result of this linear regression would be the final correct value. However, doesn't this change the final result compared to a non-linear regression? Not only that, wouldn't the result change depending on the units, because $\log(y)$ is the logarithm of a quantity which has units? If it changes the final result of the regression, which method is better to use in an experimental context such as the one mentioned above?

$\endgroup$
1
  • $\begingroup$ Sorry for the delay. I just edited my answer. Cheers :-) $\endgroup$ Sep 22, 2022 at 6:20

2 Answers 2

2
+50
$\begingroup$

Your professor is totally correct. The model $$y=a \,x^b$$ is nonlinear because of $b$. So, in a first step, you linearize $$\log(y)=\log(a)+b\log(x)=c+b\log(x)$$ and a linear regression gives the estimates of $b$ and $a=e^c$.

Now, you start with them the nonlinear regression what you must do since what is measured is $y$ and not any of its possible transforms.

Edit

When you linearize the model, the residue at point $i$ is $$\text{res}_i=\log\big[y_i^{\text{(calc)}}\big]-\log\big[y_i^{\text{(exp)}}\big]$$ which can rewrite as $$\text{res}_i=\log\Bigg[\frac{y_i^{\text{(calc)}} } {y_i^{\text{(exp)}} } \Bigg]=\log\Bigg[1+\frac{y_i^{\text{(calc)}}-y_i^{\text{(exp)}} } {y_i^{\text{(exp)}} } \Bigg]$$ So, if the error is small $$\text{res}_i \sim \frac{y_i^{\text{(calc)}}-y_i^{\text{(exp)}} } {y_i^{\text{(exp)}} }$$ which is the relative error.

In practice, when the range of the $y_i^{\text{(exp)}}$ is very large, it is often preferable to stop at this point. This is the case for physical properties like vapor pressure.

$\endgroup$
4
  • $\begingroup$ I see! Well, the problem is that my professor stated that the results of the linear regression are the final answer, and doing a nonlinear regression with the found estimates afterwards is incorrect, which, I think, is what is causing my confusion. $\endgroup$
    – ordptt
    Sep 11, 2022 at 17:12
  • $\begingroup$ @ordptt. If the data are not too noisy, the linearized model corresponds to the minimization of relative errors. If you are concerned, I can elaborate $\endgroup$ Sep 12, 2022 at 11:44
  • $\begingroup$ If possible, that would be great! $\endgroup$
    – ordptt
    Sep 12, 2022 at 17:00
  • 2
    $\begingroup$ In general the least squares idea is directly related to the maximal likelihood for the i.i.d. Gaussian noise model though it is rarely discussed in the introductory courses, so which minimization scheme is more physically relevant really depends on which noise model makes more sense. However, LSM for a linear function is computationally trivial and for a non-linear one may be a headache, so there is a tendency to linearize no matter what. @ordptt is right that linearization can change the answer from the non-linear problem but the linear LSM commutes with scaling and shifts so units are OK. $\endgroup$
    – fedja
    Sep 24, 2022 at 1:02
1
$\begingroup$

Let me expand on a couple of aspects of the previous answer that may not be obvious to a novice in curve fitting. The first is to note that a least-squares regression implicitly involves a weighting factor or uncertainty at each data point. If you can estimate the uncertainty in your data at each point and it is approximately gaussian, the statistically sound way to fit the data is to minimize $$\sum_i \left[\frac{f(x_i; a_i)-y_i}{\sigma_i}\right]^2$$ where $f(x_i; a_i)$ is your model function at the point $x_i$ with fit parameters $a_i$, $y_i$ are your measured data points, and $\sigma_i$ is the uncertainty at the point $i$.

If you ignore the weights in the intial problem (by setting them equal to 1), the weights for your transformed problem will necessarily be different leading to a different fit. For small errors, you have converted the problem from assuming absolute equal weights to one where the weights depend on the position. That's why an unweighted linear fit gives a different answer than an unweighted nonlinear fit.

If the errors are large, there is another complication. The transformed noise in the data may no longer be gaussian, even if it was close to gaussian initially. This could skew your fit so that it isn't the most likely model function to have resulted in your measured data.

Let me address the unit question directly with a concrete example. If you explictly include the units, things will cancel out nicely into dimensionless numbers. Let's say $y$ is in units of meters and $x$ is in units of seconds. Then $a$ will have to have units of meters/seconds$^b$. Treating the units like algebraic quantities, let $v = y$/meters, $\alpha = a$seconds$^b$/meters, $u=x$/seconds. The dimensions all cancel each other and the resulting equation $$v=\alpha u^b$$ has all dimensionless variables.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .