enter image description here

My teacher wanted us to try to attempt to prove this. So I noticed the summation on the left represents SST (total sum of squares) and on the right I noticed the second summation was the measure in variability of the y's in the linear regression term. However what is that first summation? Also how can I manipulate the right side to get the left side?

  • $\begingroup$ What is $\bar{Y}_i$? $\endgroup$ – David May 5 '16 at 14:54
  • $\begingroup$ I'm honestly very confused.. because that first summation poses very similar to Yi with ^ but my teacher wrote a bar instead. Do you think it was an error? $\endgroup$ – Lil May 5 '16 at 14:56
  • $\begingroup$ Yeah usually it is written $\sum_{i} (Y_i - \bar{Y})^2 = \sum_i (Y_i - \hat{Y}_i)^2 + \sum_i (\hat{Y}_i - \bar{Y})^2$ i.e. SST = SSR + SSE. Check out en.wikipedia.org/wiki/Coefficient_of_determination $\endgroup$ – David May 5 '16 at 15:01
  • $\begingroup$ yeah that's what I have noticed from online... this is to represent anova, correct? $\endgroup$ – Lil May 5 '16 at 15:01
  • $\begingroup$ robots.ox.ac.uk/~fwood/teaching/W4315_Fall2010/Lectures/… So I'm following the solution to the proof here. Do you know how they got the line after they factored the summation? $\endgroup$ – Lil May 5 '16 at 15:20

$\sum_{i=1}^n(y_i-\overline y)^2=\sum_{i=1}^n((\hat y_i-\overline y)+(y_i - \hat y_i))^2$

$=\sum_{i=1}^n((\hat y_i-\overline y)^2+2(\hat y_i-\overline y)(y_i-\hat y_i)+(y_i-\hat y_i)^2)$

$\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\overline y)$

$\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2+2\sum_{i=1}^n(y_i-\hat y_i)(\hat\beta_0+\hat\beta_1x_{i1}+\hat\beta_2x_{i2}+...+\hat\beta_mx_{im}-\overline y)$

Now let $\hat u_i=y_i-\hat y_i$

$\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2+2\sum_{i=1}^n\hat u_i(\hat\beta_0+\hat\beta_1x_{i1}+\hat\beta_2x_{i2}+...+\hat\beta_mx_{im}-\overline y)$

$\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2+2(\hat\beta_0-\overline y)\cdot \sum_{i=1}^n \hat u_i+2\hat\beta_1 \sum_{i=1}^n \hat u_ix_{i1}+2\hat\beta_2 \sum_{i=1}^n \hat u_ix_{i2}+...+2\hat\beta_m \sum_{i=1}^n \hat u_ix_{im}$

It is $\sum_{i=1}^n \hat u_i=0$ and $\sum_{i=1}^n \hat u_ix_{ij}=0 \ \ \forall j=1,2,...,m$

Finally it becomes

$\sum_{i=1}^n(y_i-\overline y)^2=\sum_{i=1}^n(\hat y_i-\overline y)^2+\sum_{i=1}^n(y_i-\hat y_i)^2$

  • $\begingroup$ In the proof, before "Finally"; Sigma1n Uihat=0 has a proof here:math.stackexchange.com/questions/494181/… I could not see why the latter holds: For all j=1,...,.m Sigma1n UihatXij=0 $\endgroup$ – Erdogan CEVHER Nov 21 '16 at 8:26
  • $\begingroup$ Now, I got it. In the same link, the partial derivative wrt to b gives the second fact above, i.e., For all j=1,...,.m Sigma1n UihatXij=0. It would be better the answer included the proofs of these 2 facts as well. $\endgroup$ – Erdogan CEVHER Nov 21 '16 at 8:48
  • $\begingroup$ @ErdoganCEVHER It seems that it wasn´t required, otherwise the OP would have asked for it. With you hints an interested reader can have a look to it. Anyway your comments are helpful. $\endgroup$ – callculus Nov 21 '16 at 9:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.