I am given the model $Y_i=\alpha_0+\beta_0 X_i+\epsilon_0$, where $i=1,2,...,n$, $X_i$ are fixed numbers and $\epsilon \sim N(0, \sigma^2).$ I am also given that $\sigma^2$ and the parameters $(\alpha_0,\beta_0)$ for $E(Y_i)=\alpha_0+\beta_0 X_i$ are unknown.

$(\alpha^*,\beta^*)$ estimate $(\alpha_0,\beta_0)$ and are found by minimizing $\sum_{i=1}^n (Y_i-\alpha-\beta X_i)^2.$

We get that $\alpha^*=\widehat{Y}-\beta^* \widehat{X}$ and $\beta^*=\frac{\sum_{i=1}^n(X_i-\widehat{X})Y_i}{\sum_{i=1}^n(X_i-\widehat{X})^2}$, where $\widehat{Y}=\frac{Y_1+Y_2+...+Y_n}{n}$ and $\widehat{X}=\frac{X_1+X_2+...+X_n}{n}$.

I have to find the distribution for $\beta^*$. I know that it has normal distribution, thus $\beta^* \sim N(E[\beta^*], var(\beta^*))$.

I think $E[\beta^*]$ is equal to $\beta_0$, but I am not entirely sure. For $var(\beta^*)$, I think it has something to do with $\epsilon \sim N(0, \sigma^2)$. Since $var(\epsilon)=\sigma^2=\sum\frac{(X_i-\widehat{X})^2}{n-1}$, I was thinking that maybe I had to substitute this into $\beta^*=\frac{\sum_{i=1}^n(X_i-\widehat{X})Y_i}{\sum_{i=1}^n(X_i-\widehat{X})^2}$ to get $var(\beta^*)$ with respect to $\sigma^2$, but I am not sure how.

An explanation would be greatly appreciated. I am having a really hard time understanding even basic concepts of this topic (and class).


You are right that $E(\beta^*) = \beta_0.$ You can show this by computing $$ E(\beta^*) = \frac{\sum_i(X_i-\widehat X) E(Y_i)}{\sum_i(X_i-\widehat X)^2}$$ and plugging in $E(Y_i) = \alpha_0 + \beta_0 X_i + E(\epsilon_i) = \alpha_0 + \beta_0 X_i$ and summing it up. (As a hint, note that $\sum_i (X_i-\widehat X) = 0$).

For the variance you have made the mistake of conflating the parameter $\sigma^2$ with its estimator $\frac{1}{n-1}\sum_i (\epsilon_i-\widehat \epsilon)^2.$ These are not equal (one is random and one isn't). (Furthermore you wrote $\sum_i (X_i -\widehat X)^2$, with $X$'s rather than with $\epsilon$'s and the sample variance of $X$ has nothing to do with $\sigma^2.)$

To get the variance of $\beta^*,$ you can use the formula $$ Var(\beta^*) = E((\beta^*-E(\beta^*))^2).$$

You can use the formula for $\beta^*$ to write this as $$ Var(\beta^*) = E\left[\left(\frac{\sum_i (X_i-\widehat X)(Y_i-E(Y_i))}{\sum_i(X_i-\widehat X)^2}\right)^2\right]=E\left[\left(\frac{\sum_i (X_i-\widehat X)\epsilon_i}{\sum_i(X_i-\widehat X)^2}\right)^2\right]$$ and it's just a matter of doing this sum using the fact that $E(\epsilon_i^2) = \sigma^2$ and $E(\epsilon_i\epsilon_j) = 0$ for $i\ne j,$

  • $\begingroup$ Thank you for your very helpful response. For $E[\beta^*]$, if $\sum_i (X_i-\widehat X) = 0$, why wouldn't the whole summation be equal to 0 or have division by 0? $\endgroup$ – Silvia Rossi Nov 12 '17 at 2:50
  • 1
    $\begingroup$ Cause that does not imply that $\sum_i (X_i-\widehat X)X_i = 0.$ In fact, it implies that $\sum_i (X_i-\widehat X)X_i = \sum_i (X_i-\widehat X)^2.$ $\endgroup$ – spaceisdarkgreen Nov 12 '17 at 3:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.