Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to fit a set of measured data $x_i$ and $y_i$ to the expression: $$ \beta^2 + \beta x_i = y_i $$ $\beta$ is my only free parameter. Although this is a really simple expression, the standard approaches of linear regression do not hold because of the $\beta^2$ term. But I can calculate a $\hat\beta$ that minimizes the squared error: $$ \varepsilon_i = \beta^2 + \beta x_i - y_i \\ \frac{1}{2}\frac{\partial}{\partial x_i}\sum \varepsilon_i^2 = \sum \varepsilon_i \beta = 0 \\ -\frac{1}{2}\frac{\partial}{\partial y_i}\sum \varepsilon_i^2 = \sum \varepsilon_i = 0 \\ \Rightarrow \qquad \hat\beta_{1/2}=-\frac{\langle x\rangle}{2}\pm \sqrt{\frac{\langle x \rangle}{4}+\langle y \rangle} $$ (Obviously the derivative with respect to $x_i$ does not give any useful information.)

My actual question is, how I can estimate the standard deviation of my least squares estimate $\hat\beta$ according to the input data $x_i$ and $y_i$. I tried to find some analogy to the linear regression case, but I am completely lost as the expression for $\hat\beta$ is so nonlinear...

Also, it seems that this is quite a trivial standard case of a nonlinear regression. However, I was not able to find any recipe on how to solve this while searching for terms like "second order regression one parameter" and the like on the net. Am I missing something? Is there a simple way to reformulate this as a first order fit problem?

share|cite|improve this question
up vote 1 down vote accepted

You should take the derivative of $$ \sum_{i=1}^N \epsilon_i^2 $$ with respect to $\beta$, not $x_i$. When you do this, you get $$ \begin{eqnarray} 0 &=& \frac{\partial}{\partial \beta} \sum_{i=1}^N \epsilon_i^2 = 2 \sum_{i=1}^N \epsilon_i \frac{\partial \epsilon_i}{\partial \beta} = 2 \sum_{i=1}^N \left(\beta^2 + \beta x_i - y_i\right) \left(2 \beta + x_i\right) \\ 0 &=& \sum_{i=1}^N \left(2 \beta^3 + 3 \beta^2 x_i - 2 \beta y_i + \beta x_i^2 - x_i y_i\right) \\ 0 &=& 2 \beta^3 + 3 \beta^2 \left< x \right> - 2 \beta \left< y \right> + \beta \left< x^2 \right> - \left<x y\right> \end{eqnarray} $$ where $\left<z\right>$ is the average value of $z_i$. Now you have to solve this cubic for $\beta$.

share|cite|improve this answer
Oh my goodness. You're so right :O – Jakob S. Oct 23 '12 at 14:41

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.