Bias and variance of two alternative estimators

Question

Apologies in advance as this seems like quite a simple problem but I have hit a wall and could use some guidance on next steps. Here is the problem and then where I am at:

Suppose that the random variables $Y_1,...,Y_n$ satisfy $$Y_i=\beta x_i+\epsilon_i$$ $$i=1,...,n$$ where $x_1,...,x_n$ are deterministic and strictly positive scalars, $\epsilon_1,...,\epsilon_n$ are iid n$(0,\sigma^2)$ and $\sigma^2$ is unknown. Consider two different estimators: $\hat\theta_1=\frac{\sum_i^n Y_i}{\sum_i^n x_i}$ and $\hat\theta_2=\frac{1}{n}\sum_i^n\frac{Y_i}{x_i}$.

1. What is the bias of $\hat\theta_1$ and $\hat\theta_2$?
2. What is the variance of $\hat\theta_1$ and $\hat\theta_2$?
3. Which estimator has a lower variance? Use Jensen's inequality
4. Which estimator do you prefer if you want to minimize the mean squared error?

For bias, I realize I am looking to express E$(\hat\theta_1)-\theta$ and E$(\hat\theta_2)-\theta$, where $\theta$ is the true value of $\beta$. However, can I simply write out those expressions (e.g. Bias$(\hat\theta_1)=$ E$(\frac{\bar Y}{\bar x})-\beta$) or can I take steps to simplify? Intuitively I would expect both estimators to be unbiased but I'm not sure what assumptions/properties I need to show in order to prove that.

I also am worried I am missing some implication regarding the deterministic scalars comment and the description of $\epsilon$.

For variance I think I will have a better idea once I know how to go about the bias question. But any tips on this will help. Then I understand how to choose the preferred option once I have bias and variance.

Thanks so much for any help!

Answer 1

1. Since $\displaystyle \hat\theta_1=\frac{\sum_{i=1}^n Y_i}{\sum_{i=1}^n x_i}$, $$E(\hat \theta_1) = \frac{1}{\sum_{i=1}^n x_i}\sum_{i=1}^n E(Y_i)=\frac{1}{\sum_{i=1}^n x_i}\sum_{i=1}^n \beta x_i = \beta$$

Since $\displaystyle \hat\theta_2=\frac{1}{n}\sum_{i=1}^n\frac{Y_i}{x_i}$, $$E(\hat \theta_2) =\frac 1n \sum_{i=1}^n\frac{1}{x_i}E(Y_i)=\frac 1n \sum_{i=1}^n\frac{1}{x_i}\beta x_i = \beta$$

These estimators are unbiased.

2. $$\begin{aligned}[t] Var(\hat\theta_1) &= E[(\hat\theta_1-\beta )^2]=E\left[\left(\frac{\sum_{i=1}^n Y_i-x_i\beta }{\sum_{i=1}^n x_i} \right)^2 \right]\\ &= \frac{1}{\left(\sum_{i=1}^n x_i\right)^2} E\left[\left(\sum_{i=1}^n Y_i-x_i\beta \right)^2 \right]\\ &= \frac{1}{\left(\sum_{i=1}^n x_i\right)^2} E\left[\left( \sum_{i=1}^n\epsilon_i\right)^2 \right] \end{aligned}$$

Since the $\epsilon_i$ are independent, they're uncorrelated, so $E\left[\left( \sum_{i=1}^n\epsilon_i\right)^2\right] = \sum_{i=1}^n E(\epsilon_i^2)=n\sigma^2$.

Thus $$Var(\hat\theta_1) = \frac{n\sigma^2}{\left(\sum_{i=1}^n x_i\right)^2}$$

$$\begin{aligned}[t] Var(\hat\theta_2) &=E[(\hat\theta_2-\beta )^2]= E\left[\left(\frac 1n\sum_{i=1}^n \frac{Y_i-x_i\beta}{x_i} \right)^2 \right]\\ &=\frac 1{n^2}E\left[\left(\sum_{i=1}^n \frac{\epsilon_i}{x_i} \right)^2 \right]\\ &= \frac 1{n^2}\sum_{i=1}^n \frac{E[\epsilon_i^2]}{x_i^2} \\ &= \frac {\sigma^2}{n^2}\sum_{i=1}^n \frac 1{x_i^2} \end{aligned}$$

3. With Jensen applied to $x\mapsto \frac{1}{x^2}$ which is convex over the positive real numbers, $$\sum_{i=1}^n \frac 1n \frac{1}{x_i^2} \geq \frac {1}{\left( \sum_{i=1}^n \frac{1}n x_i\right)^2}$$ which easily rewrites as $$Var(\hat\theta_2)\geq Var(\hat\theta_1)$$

4. Since both estimators are unbiased, the MSE is the variance, hence $\hat\theta_1$ is preferable.