Second order least squares problem with one parameter

Question

I want to fit a set of measured data $x_i$ and $y_i$ to the expression: $$ \beta^2 + \beta x_i = y_i $$ $\beta$ is my only free parameter. Although this is a really simple expression, the standard approaches of linear regression do not hold because of the $\beta^2$ term. But I can calculate a $\hat\beta$ that minimizes the squared error: $$ \varepsilon_i = \beta^2 + \beta x_i - y_i \\ \frac{1}{2}\frac{\partial}{\partial x_i}\sum \varepsilon_i^2 = \sum \varepsilon_i \beta = 0 \\ -\frac{1}{2}\frac{\partial}{\partial y_i}\sum \varepsilon_i^2 = \sum \varepsilon_i = 0 \\ \Rightarrow \qquad \hat\beta_{1/2}=-\frac{\langle x\rangle}{2}\pm \sqrt{\frac{\langle x \rangle}{4}+\langle y \rangle} $$ (Obviously the derivative with respect to $x_i$ does not give any useful information.)

My actual question is, how I can estimate the standard deviation of my least squares estimate $\hat\beta$ according to the input data $x_i$ and $y_i$. I tried to find some analogy to the linear regression case, but I am completely lost as the expression for $\hat\beta$ is so nonlinear...

Also, it seems that this is quite a trivial standard case of a nonlinear regression. However, I was not able to find any recipe on how to solve this while searching for terms like "second order regression one parameter" and the like on the net. Am I missing something? Is there a simple way to reformulate this as a first order fit problem?

Answer 1

You should take the derivative of $$ \sum_{i=1}^N \epsilon_i^2 $$ with respect to $\beta$, not $x_i$. When you do this, you get $$ \begin{eqnarray} 0 &=& \frac{\partial}{\partial \beta} \sum_{i=1}^N \epsilon_i^2 = 2 \sum_{i=1}^N \epsilon_i \frac{\partial \epsilon_i}{\partial \beta} = 2 \sum_{i=1}^N \left(\beta^2 + \beta x_i - y_i\right) \left(2 \beta + x_i\right) \\ 0 &=& \sum_{i=1}^N \left(2 \beta^3 + 3 \beta^2 x_i - 2 \beta y_i + \beta x_i^2 - x_i y_i\right) \\ 0 &=& 2 \beta^3 + 3 \beta^2 \left< x \right> - 2 \beta \left< y \right> + \beta \left< x^2 \right> - \left<x y\right> \end{eqnarray} $$ where $\left<z\right>$ is the average value of $z_i$. Now you have to solve this cubic for $\beta$.