1
$\begingroup$

I am trying to apply the second partial derivative test to show that the simple least square estimators $\hat\beta_0$ and $\hat\beta_1$ does minimize the sum of the squared errors based on page 3 of this lecture notes. Based on the second last equations on page 3, I found that:

$\frac{\partial^2 SSE}{\partial \hat\beta_0^2} = 2n > 0 $, which should show that this is a minimum if the second condition holds true. However, in trying to evaluate the second condition, I ended up with:

$2n*2{\sum}x^2 - 4({\sum}x)^2$. I'm unsure if I did something wrong and if not, how do I prove that this is greater than zero?

$\endgroup$

1 Answer 1

1
$\begingroup$

It seems that you are on the right track. You can divide both sides of the equation by 4 and $n^2$.

$$\frac1n{\sum_{i=1}^n}x_i^2 - \frac1{n^2}\left({\sum_{i=1}^n}x_i\right)^2$$

This is the definition of the (empirical) $\text{variance}$ for $n$ values, which is always larger than $0$ (at least one $x_i\neq \overline x$). It can be further transformed to get a more familiar expression. We use that $\frac1{n^2}\left({\sum_{i=1}^n}x_i\right)^2=\left(\frac1{n}{\sum_{i=1}^n}x_i\right)^2=\overline x^2$

$$\frac1n{\sum_{i=1}^n}x_i^2 - \overline x^2$$

Adding & subtracting $2\overline x^2$

$$\frac1n{\sum_{i=1}^n}x_i^2 \underbrace{- 2\overline x^2+2\overline x^2}_{=0}- \overline x^2$$

$$\frac1n{\sum_{i=1}^n}x_i^2 - 2\overline x^2+\overline x^2$$

$$\frac1n{\sum_{i=1}^n}x_i^2 - 2\left(\frac1n \sum_{i=1}^{n} x_i\right) \overline x+\overline x^2$$

$$\frac1n{\sum_{i=1}^n}x_i^2 - 2\frac1n \sum_{i=1}^{n} x_i\overline x++\overline x^2$$

$$\frac1n\sum_{i=1}^{n} \left(x_i^2-2x_i\overline x+\overline x^2\right)$$

Using the first binomial fomumla we get another common version of the variance

$$\frac1n\sum_{i=1}^{n} \left(x_i-\overline x\right)^2>0$$

A square of a number is always positive if the squared number is not $0$. Again $x_i\neq \overline x$ in at least one case.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .