# Show if the estimator is unbiased

Suppose that the random variables $Y_1... Y_n$ satisfy $Y_i = \beta x_i + ϵ_i$ , i = 1...n where $x_i$ are fixed constants and the $ϵ_i$ are iid Normally distributed random variables with mean zero and variance $\sigma^2$

I need to show if the estimator $$\beta_a = \frac{\sum_{i=1}^n x_iY_i}{\sum_{i=1}^n x_i^2}$$ is an unbiased estimator of $\beta$ or not. I understand that I have to show that $E(\beta_a) = \beta$ for $\beta_a$ to be an unbiased estimator of $\beta$. I have first simplified $\beta_a$ by replacing $Y_i$ with $\beta x_i + ϵ_i$ ,used some basic summation properties and took $E(\beta_a)$and arrived at the following equation: $$E(\beta_a) = \beta + \frac{E(\sum_{i=1}^n x_iϵ_i)}{\sum_{i=1}^n x_i^2}$$

Not sure how to proceed after this. I also need to find the variance of the estimator $\beta_a$.

• The numerator in the second equation should have $\epsilon_i$ Jun 8 '18 at 7:36
• Thanks, I didn't realize, it was a typo. I have edited it. Any idea how to proceed? Jun 8 '18 at 7:52
• What's $x$? It looks like you probably mean $x_i$ where it says $x$ in the denominators? Jun 8 '18 at 7:53
• You can't find the variance without knowing the covariances of the $Y_i$. Perhaps you forgot to state the premise that they're independent? Jun 8 '18 at 7:56
• Thanks, yes I forgot the subscript i's for the x. No there is no premise that $Y_i$' s are independent. Jun 8 '18 at 8:04

$$E\left(\sum_{i=1}^nx_i\epsilon_i\right)=\sum_{i=1}^nx_iE\left(\epsilon_i\right)=0\;.$$

• How does $\sum_{i=1}^n E( x_iϵ_i )= \sum_{i=1}^n x_iE(ϵ_i)$ ? Jun 8 '18 at 8:13
• @StatQ $x_i$ is a fixed constant. For every fixed constant $c$ and random variable $X$ we have $\mathbb EcX=c\mathbb EX$ (if the mean exists, of course). Jun 8 '18 at 8:38
• @drhab Thanks, I should have realized that. Got confused because of the subscript i. Jun 8 '18 at 8:44

Since $E(\beta_a) = \beta$ from first part, I have managed to find the solution to the variance. I used the equation $$Var(\beta_a) = E(\beta_a^2) - (E(\beta_s))^2 = E(\beta_a^2) - \beta^2$$ and using summation rules + expectation rules ended up with the solution: $Var(\beta_a) = \sigma^2$

Please correct me if wrong. Thanks

• As I noted under the question, you cannot know the variance without knowing the covariances of the $Y_i$ (or equivalently the $\epsilon_i$) or making an assumption about their independence. The term $E\left(\beta_a^2\right)$ depends on these covariances. Jun 8 '18 at 14:05
• @joriki Thanks. According to the question the $ϵ_i$'s are independent (iid). It does not state independence of $Y_i$'s. However, since Y depends on $x_i$'s and $ϵ_i$'s only, where $x_i$'s are constant and $ϵ_i$'s are independent, does this not make $Y_i$'s independent? Jun 8 '18 at 14:15
• I'm sorry, I'd overlooked that the question does state that the $\epsilon_i$ are independent. Yes, that does make the $Y_i$ independent. Jun 8 '18 at 14:23
• I think you're missing a factor of $\sum x_i^2$ in the denominator. There are two of those in the denominator and only one in the numerator. Also intuitively, it makes sense that the estimator for $\beta$ will be more uncertain the smaller the $x_i$ -- in the extreme case $x_i\to0$, you know almost nothing about $\beta$ and your estimator is merely amplifying the fluctuations in the $\epsilon_i$. Jun 8 '18 at 14:28
• @joriki Got it!! I had missed squaring the denominator in one of the intermediate steps. Yes it makes sense.. Thanks so much for the help. Jun 8 '18 at 23:46