2
$\begingroup$

I found the following proof in my notes:

$E(Y_i) = E[\beta_0 + \beta X_i + \varepsilon_i] =\cdots= \beta_0 + \beta X_i$. This does not seem right to me, however. Why would $E(\beta_1 X_i) = \beta_1 X_i$? I wonder if i might have written it down incorrectly, with the actual proof meaning to be for the estimated value Yi hat (I don't know how to code this unfortunately). Does anyone recall this property of linear regression?

$\endgroup$
  • $\begingroup$ Perhaps you meant $\beta_i E X_i$? Your question needs some context... $\endgroup$ – copper.hat May 19 '13 at 15:56
  • $\begingroup$ It seems like what we are trying to show is that the expectation of a linear regression is a linear function of expectations but we may also be trying to show the proof shown on page 4 here: web.njit.edu/~wguo/Math644_2012/Math644_Chapter%201_part2.pdf $\endgroup$ – user78504 May 19 '13 at 16:06
  • $\begingroup$ That proof shows the expectation of $Y_{i}$ is equivalent to the expectation of its estimate Yi hat $\endgroup$ – user78504 May 19 '13 at 16:07
  • $\begingroup$ I actually think my answer is a bit more complete than the "accepted" one. $\endgroup$ – Michael Hardy May 19 '13 at 19:00
0
$\begingroup$

In this sort of regression problem, $X_i$ may be random in the sense that if you take another sample, all the $X_i$ values change, but one behaves as if one seeks the conditional expected value of $Y_i$ given $X_i$, so that in effect $X_i$ is treated as if it were constant rather than random. And the $\beta$s are also being treated as constant.

$\endgroup$
  • $\begingroup$ Right, so in fact I was correct since it is in effect being treated as a constant. I misunderstood before. $\endgroup$ – user78504 May 23 '13 at 16:06
2
$\begingroup$

If $Y_i=\beta_0+\beta X_i+\epsilon_i$, where $\beta_0$ and $\beta$ are constants and $\epsilon_i$ is an "error" random variable with mean $0$, then $E(Y_i)=\beta_0+\beta E(X_i)$.

$\endgroup$
  • $\begingroup$ This is what I thought, and what copper-hat mentioned above. It seems likely I missed the E. Thank you very much. While I'm here, could I also ask: Why is it that the error has to be 'centered' or mean zero, and what is the importance of this property? Do you think it is just to say 'remember, linearity of expectation works here too'? $\endgroup$ – user78504 May 19 '13 at 16:10
  • $\begingroup$ If there is a systematic error, we would probably incorporate it in the $\beta_0$ term. Linearity of expectation would work in any case, we just use $E(\epsilon_i)$. $\endgroup$ – André Nicolas May 19 '13 at 16:15
  • $\begingroup$ That makes sense, thanks. My other final question is: why in the linked document above, or in general, is E(Y) = E(Y'), that is, the estimated Y's expectation is the same as Y's expectation? Is it that the variance is what's changing not the mean? $\endgroup$ – user78504 May 19 '13 at 16:26
  • $\begingroup$ The least squares estimator is unbiased. That takes some proving. $\endgroup$ – André Nicolas May 19 '13 at 16:45
  • $\begingroup$ Ohh, so that's what's meant by unbiased essentially. The expected value of the estimate is unchanged. $\endgroup$ – user78504 May 19 '13 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.