Suppose that the random variables $Y_1... Y_n$ satisfy $Y_i = \beta x_i + ϵ_i$ , i = 1...n where $x_i$ are fixed constants and the $ϵ_i$ are iid Normally distributed random variables with mean zero and variance $\sigma^2$

I need to show if the estimator $$\beta_a = \frac{\sum_{i=1}^n x_iY_i}{\sum_{i=1}^n x_i^2}$$ is an unbiased estimator of $\beta$ or not. I understand that I have to show that $E(\beta_a) = \beta$ for $\beta_a $ to be an unbiased estimator of $\beta$. I have first simplified $\beta_a$ by replacing $Y_i$ with $\beta x_i + ϵ_i$ ,used some basic summation properties and took $E(\beta_a) $and arrived at the following equation: $$E(\beta_a) = \beta + \frac{E(\sum_{i=1}^n x_iϵ_i)}{\sum_{i=1}^n x_i^2}$$

Not sure how to proceed after this. I also need to find the variance of the estimator $\beta_a$.

  • $\begingroup$ The numerator in the second equation should have $\epsilon_i$ $\endgroup$ – TenaliRaman Jun 8 '18 at 7:36
  • $\begingroup$ Thanks, I didn't realize, it was a typo. I have edited it. Any idea how to proceed? $\endgroup$ – StatQ Jun 8 '18 at 7:52
  • $\begingroup$ What's $x$? It looks like you probably mean $x_i$ where it says $x$ in the denominators? $\endgroup$ – joriki Jun 8 '18 at 7:53
  • $\begingroup$ You can't find the variance without knowing the covariances of the $Y_i$. Perhaps you forgot to state the premise that they're independent? $\endgroup$ – joriki Jun 8 '18 at 7:56
  • $\begingroup$ Thanks, yes I forgot the subscript i's for the x. No there is no premise that $Y_i$' s are independent. $\endgroup$ – StatQ Jun 8 '18 at 8:04

By linearity of expectation,

$$ E\left(\sum_{i=1}^nx_i\epsilon_i\right)=\sum_{i=1}^nx_iE\left(\epsilon_i\right)=0\;. $$

  • $\begingroup$ How does $\sum_{i=1}^n E( x_iϵ_i )= \sum_{i=1}^n x_iE(ϵ_i)$ ? $\endgroup$ – StatQ Jun 8 '18 at 8:13
  • $\begingroup$ @StatQ $x_i$ is a fixed constant. For every fixed constant $c$ and random variable $X$ we have $\mathbb EcX=c\mathbb EX$ (if the mean exists, of course). $\endgroup$ – drhab Jun 8 '18 at 8:38
  • $\begingroup$ @drhab Thanks, I should have realized that. Got confused because of the subscript i. $\endgroup$ – StatQ Jun 8 '18 at 8:44

Since $E(\beta_a) = \beta $ from first part, I have managed to find the solution to the variance. I used the equation $$Var(\beta_a) = E(\beta_a^2) - (E(\beta_s))^2 = E(\beta_a^2) - \beta^2$$ and using summation rules + expectation rules ended up with the solution: $Var(\beta_a) = \sigma^2$

Please correct me if wrong. Thanks

  • $\begingroup$ As I noted under the question, you cannot know the variance without knowing the covariances of the $Y_i$ (or equivalently the $\epsilon_i$) or making an assumption about their independence. The term $E\left(\beta_a^2\right)$ depends on these covariances. $\endgroup$ – joriki Jun 8 '18 at 14:05
  • $\begingroup$ @joriki Thanks. According to the question the $ϵ_i$'s are independent (iid). It does not state independence of $Y_i$'s. However, since Y depends on $x_i$'s and $ϵ_i$'s only, where $x_i$'s are constant and $ϵ_i$'s are independent, does this not make $Y_i$'s independent? $\endgroup$ – StatQ Jun 8 '18 at 14:15
  • $\begingroup$ I'm sorry, I'd overlooked that the question does state that the $\epsilon_i$ are independent. Yes, that does make the $Y_i$ independent. $\endgroup$ – joriki Jun 8 '18 at 14:23
  • $\begingroup$ I think you're missing a factor of $\sum x_i^2$ in the denominator. There are two of those in the denominator and only one in the numerator. Also intuitively, it makes sense that the estimator for $\beta$ will be more uncertain the smaller the $x_i$ -- in the extreme case $x_i\to0$, you know almost nothing about $\beta$ and your estimator is merely amplifying the fluctuations in the $\epsilon_i$. $\endgroup$ – joriki Jun 8 '18 at 14:28
  • 1
    $\begingroup$ @joriki Got it!! I had missed squaring the denominator in one of the intermediate steps. Yes it makes sense.. Thanks so much for the help. $\endgroup$ – StatQ Jun 8 '18 at 23:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.