# Distribution of point estimator

I am given the model $Y_i=\alpha_0+\beta_0 X_i+\epsilon_0$, where $i=1,2,...,n$, $X_i$ are fixed numbers and $\epsilon \sim N(0, \sigma^2).$ I am also given that $\sigma^2$ and the parameters $(\alpha_0,\beta_0)$ for $E(Y_i)=\alpha_0+\beta_0 X_i$ are unknown.

$(\alpha^*,\beta^*)$ estimate $(\alpha_0,\beta_0)$ and are found by minimizing $\sum_{i=1}^n (Y_i-\alpha-\beta X_i)^2.$

We get that $\alpha^*=\widehat{Y}-\beta^* \widehat{X}$ and $\beta^*=\frac{\sum_{i=1}^n(X_i-\widehat{X})Y_i}{\sum_{i=1}^n(X_i-\widehat{X})^2}$, where $\widehat{Y}=\frac{Y_1+Y_2+...+Y_n}{n}$ and $\widehat{X}=\frac{X_1+X_2+...+X_n}{n}$.

I have to find the distribution for $\beta^*$. I know that it has normal distribution, thus $\beta^* \sim N(E[\beta^*], var(\beta^*))$.

I think $E[\beta^*]$ is equal to $\beta_0$, but I am not entirely sure. For $var(\beta^*)$, I think it has something to do with $\epsilon \sim N(0, \sigma^2)$. Since $var(\epsilon)=\sigma^2=\sum\frac{(X_i-\widehat{X})^2}{n-1}$, I was thinking that maybe I had to substitute this into $\beta^*=\frac{\sum_{i=1}^n(X_i-\widehat{X})Y_i}{\sum_{i=1}^n(X_i-\widehat{X})^2}$ to get $var(\beta^*)$ with respect to $\sigma^2$, but I am not sure how.

An explanation would be greatly appreciated. I am having a really hard time understanding even basic concepts of this topic (and class).

You are right that $E(\beta^*) = \beta_0.$ You can show this by computing $$E(\beta^*) = \frac{\sum_i(X_i-\widehat X) E(Y_i)}{\sum_i(X_i-\widehat X)^2}$$ and plugging in $E(Y_i) = \alpha_0 + \beta_0 X_i + E(\epsilon_i) = \alpha_0 + \beta_0 X_i$ and summing it up. (As a hint, note that $\sum_i (X_i-\widehat X) = 0$).
For the variance you have made the mistake of conflating the parameter $\sigma^2$ with its estimator $\frac{1}{n-1}\sum_i (\epsilon_i-\widehat \epsilon)^2.$ These are not equal (one is random and one isn't). (Furthermore you wrote $\sum_i (X_i -\widehat X)^2$, with $X$'s rather than with $\epsilon$'s and the sample variance of $X$ has nothing to do with $\sigma^2.)$
To get the variance of $\beta^*,$ you can use the formula $$Var(\beta^*) = E((\beta^*-E(\beta^*))^2).$$
You can use the formula for $\beta^*$ to write this as $$Var(\beta^*) = E\left[\left(\frac{\sum_i (X_i-\widehat X)(Y_i-E(Y_i))}{\sum_i(X_i-\widehat X)^2}\right)^2\right]=E\left[\left(\frac{\sum_i (X_i-\widehat X)\epsilon_i}{\sum_i(X_i-\widehat X)^2}\right)^2\right]$$ and it's just a matter of doing this sum using the fact that $E(\epsilon_i^2) = \sigma^2$ and $E(\epsilon_i\epsilon_j) = 0$ for $i\ne j,$
• Thank you for your very helpful response. For $E[\beta^*]$, if $\sum_i (X_i-\widehat X) = 0$, why wouldn't the whole summation be equal to 0 or have division by 0? – Silvia Rossi Nov 12 '17 at 2:50
• Cause that does not imply that $\sum_i (X_i-\widehat X)X_i = 0.$ In fact, it implies that $\sum_i (X_i-\widehat X)X_i = \sum_i (X_i-\widehat X)^2.$ – spaceisdarkgreen Nov 12 '17 at 3:03