I want to fit a set of measured data $x_i$ and $y_i$ to the expression: $$ \beta^2 + \beta x_i = y_i $$ $\beta$ is my only free parameter. Although this is a really simple expression, the standard approaches of linear regression do not hold because of the $\beta^2$ term. But I can calculate a $\hat\beta$ that minimizes the squared error: $$ \varepsilon_i = \beta^2 + \beta x_i - y_i \\ \frac{1}{2}\frac{\partial}{\partial x_i}\sum \varepsilon_i^2 = \sum \varepsilon_i \beta = 0 \\ -\frac{1}{2}\frac{\partial}{\partial y_i}\sum \varepsilon_i^2 = \sum \varepsilon_i = 0 \\ \Rightarrow \qquad \hat\beta_{1/2}=-\frac{\langle x\rangle}{2}\pm \sqrt{\frac{\langle x \rangle}{4}+\langle y \rangle} $$ (Obviously the derivative with respect to $x_i$ does not give any useful information.)

My actual question is, how I can estimate the standard deviation of my least squares estimate $\hat\beta$ according to the input data $x_i$ and $y_i$. I tried to find some analogy to the linear regression case, but I am completely lost as the expression for $\hat\beta$ is so nonlinear...

Also, it seems that this is quite a trivial standard case of a nonlinear regression. However, I was not able to find any recipe on how to solve this while searching for terms like "second order regression one parameter" and the like on the net. Am I missing something? Is there a simple way to reformulate this as a first order fit problem?


You should take the derivative of $$ \sum_{i=1}^N \epsilon_i^2 $$ with respect to $\beta$, not $x_i$. When you do this, you get $$ \begin{eqnarray} 0 &=& \frac{\partial}{\partial \beta} \sum_{i=1}^N \epsilon_i^2 = 2 \sum_{i=1}^N \epsilon_i \frac{\partial \epsilon_i}{\partial \beta} = 2 \sum_{i=1}^N \left(\beta^2 + \beta x_i - y_i\right) \left(2 \beta + x_i\right) \\ 0 &=& \sum_{i=1}^N \left(2 \beta^3 + 3 \beta^2 x_i - 2 \beta y_i + \beta x_i^2 - x_i y_i\right) \\ 0 &=& 2 \beta^3 + 3 \beta^2 \left< x \right> - 2 \beta \left< y \right> + \beta \left< x^2 \right> - \left<x y\right> \end{eqnarray} $$ where $\left<z\right>$ is the average value of $z_i$. Now you have to solve this cubic for $\beta$.

  • $\begingroup$ Oh my goodness. You're so right :O $\endgroup$ – Jakob S. Oct 23 '12 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.