1
$\begingroup$

I am trying to apply the second partial derivative test to show that the simple least square estimators $\hat\beta_0$ and $\hat\beta_1$ does minimize the sum of the squared errors based on page 3 of this lecture notes. Based on the second last equations on page 3, I found that:

$\frac{\partial^2 SSE}{\partial \hat\beta_0^2} = 2n > 0 $, which should show that this is a minimum if the second condition holds true. However, in trying to evaluate the second condition, I ended up with:

$2n*2{\sum}x^2 - 4({\sum}x)^2$. I'm unsure if I did something wrong and if not, how do I prove that this is greater than zero?

$\endgroup$
1
$\begingroup$

It seems that you are on the right track. You can divide both sides of the equation by 4 and $n^2$.

$$\frac1n{\sum_{i=1}^n}x_i^2 - \frac1{n^2}\left({\sum_{i=1}^n}x_i\right)^2$$

This is the definition of the (empirical) $\text{variance}$ for $n$ values, which is always larger than $0$ (at least one $x_i\neq \overline x$). It can be further transformed to get a more familiar expression. We use that $\frac1{n^2}\left({\sum_{i=1}^n}x_i\right)^2=\left(\frac1{n}{\sum_{i=1}^n}x_i\right)^2=\overline x^2$

$$\frac1n{\sum_{i=1}^n}x_i^2 - \overline x^2$$

Adding at subtracting $2\overline x^2$

$$\frac1n{\sum_{i=1}^n}x_i^2 \underbrace{- 2\overline x^2+2\overline x^2}_{=0}- \overline x^2$$

$$\frac1n{\sum_{i=1}^n}x_i^2 - 2\overline x^2+\overline x^2$$

$$\frac1n{\sum_{i=1}^n}x_i^2 - 2\left(\frac1n \sum_{i=1}^{n} x_i\right) \overline x+\overline x^2$$

$$\frac1n{\sum_{i=1}^n}x_i^2 - 2\frac1n \sum_{i=1}^{n} x_i\overline x++\overline x^2$$

$$\frac1n\sum_{i=1}^{n} \left(x_i^2-2x_i\overline x+\overline x^2\right)$$

Using the first binomial fomumla we get another common version of the variance

$$\frac1n\sum_{i=1}^{n} \left(x_i-\overline x\right)^2>0$$

A square of a number is always positive if the squared number is not $0$. Again $x_i\neq \overline x$ in at least one case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.