# Linear regression proof

My teacher wanted us to try to attempt to prove this. So I noticed the summation on the left represents SST (total sum of squares) and on the right I noticed the second summation was the measure in variability of the y's in the linear regression term. However what is that first summation? Also how can I manipulate the right side to get the left side?

• What is $\bar{Y}_i$? – David May 5 '16 at 14:54
• I'm honestly very confused.. because that first summation poses very similar to Yi with ^ but my teacher wrote a bar instead. Do you think it was an error? – Lil May 5 '16 at 14:56
• Yeah usually it is written $\sum_{i} (Y_i - \bar{Y})^2 = \sum_i (Y_i - \hat{Y}_i)^2 + \sum_i (\hat{Y}_i - \bar{Y})^2$ i.e. SST = SSR + SSE. Check out en.wikipedia.org/wiki/Coefficient_of_determination – David May 5 '16 at 15:01
• yeah that's what I have noticed from online... this is to represent anova, correct? – Lil May 5 '16 at 15:01
• robots.ox.ac.uk/~fwood/teaching/W4315_Fall2010/Lectures/… So I'm following the solution to the proof here. Do you know how they got the line after they factored the summation? – Lil May 5 '16 at 15:20

$\sum_{i=1}^n(y_i-\overline y)^2=\sum_{i=1}^n((\hat y_i-\overline y)+(y_i - \hat y_i))^2$

$=\sum_{i=1}^n((\hat y_i-\overline y)^2+2(\hat y_i-\overline y)(y_i-\hat y_i)+(y_i-\hat y_i)^2)$

$\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\overline y)$

$\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2+2\sum_{i=1}^n(y_i-\hat y_i)(\hat\beta_0+\hat\beta_1x_{i1}+\hat\beta_2x_{i2}+...+\hat\beta_mx_{im}-\overline y)$

Now let $\hat u_i=y_i-\hat y_i$

$\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2+2\sum_{i=1}^n\hat u_i(\hat\beta_0+\hat\beta_1x_{i1}+\hat\beta_2x_{i2}+...+\hat\beta_mx_{im}-\overline y)$

$\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2+2(\hat\beta_0-\overline y)\cdot \sum_{i=1}^n \hat u_i+2\hat\beta_1 \sum_{i=1}^n \hat u_ix_{i1}+2\hat\beta_2 \sum_{i=1}^n \hat u_ix_{i2}+...+2\hat\beta_m \sum_{i=1}^n \hat u_ix_{im}$

It is $\sum_{i=1}^n \hat u_i=0$ and $\sum_{i=1}^n \hat u_ix_{ij}=0 \ \ \forall j=1,2,...,m$

Finally it becomes

$\sum_{i=1}^n(y_i-\overline y)^2=\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2$

• In the proof, before "Finally"; Sigma1n Uihat=0 has a proof here:math.stackexchange.com/questions/494181/… I could not see why the latter holds: For all j=1,...,.m Sigma1n UihatXij=0 – Erdogan CEVHER Nov 21 '16 at 8:26
• Now, I got it. In the same link, the partial derivative wrt to b gives the second fact above, i.e., For all j=1,...,.m Sigma1n UihatXij=0. It would be better the answer included the proofs of these 2 facts as well. – Erdogan CEVHER Nov 21 '16 at 8:48
• @ErdoganCEVHER It seems that it wasn´t required, otherwise the OP would have asked for it. With you hints an interested reader can have a look to it. Anyway your comments are helpful. – callculus Nov 21 '16 at 9:13