$\hat \theta_1$ and $\hat \theta_2$ are independent unbiased estimators of $\theta$ with $Var[\hat \theta_2] = 3Var[\hat \theta_1]$.

For what values of $k_1$ and $k_2$ will the combined estimator $k_1\hat \theta_1 + k_2\hat \theta_2$ be an unbiased estimator with smallest variance amongst all such linear combinations?

My attempt: I started by establishing $$E[\hat \theta_1] = E[\hat \theta_2] = \theta$$ Next, I wrote $$E[k_1\hat \theta_1 + k_2\hat \theta_2] = \theta \quad (*)$$ (*) this summation is correct since they're independent and unbiased estimators, yes?

I then split this with linearity properties.. $$E[k_1\hat \theta_1] + E[k_2\hat \theta_2] = \theta$$ $$k_1(E[\hat \theta_1]) + k_2(E[\hat \theta_2]) = \theta$$ $$k_1(\theta) + k_2(\theta) = \theta$$ $$k_1 + k_2 = 1$$

Now I know that the sum of the $k$ constants is 1.. And I'm assuming that I need to use the $\theta$ Variance relationship given to me in the question to find their individual values but when I tried it, I didn't really get anywhere.

I wrote: $$Var[\hat \theta] = E[\hat \theta^2] - (E[\hat \theta])^2$$ Given: $$ Var[\hat \theta_2] = 3Var[\hat \theta_1]$$ $$E[\hat \theta^2_2] - (E[\hat \theta_2])^2 = 3(E[\hat \theta^2_1] - (E[\hat \theta_1])^2)$$ $$E[\hat \theta^2_2] - \theta^2 = 3E[\hat \theta^2_1] - 3\theta^2$$ $$2\theta^2 = 2E[\hat \theta^2_1] \qquad (**)$$ $$\theta^2 = E[\hat \theta^2_1]$$ ??? Where do I go from here?

(**) I'm not even sure this is right, I just assumed since $$E[\hat \theta_1] = E[\hat \theta_2]$$ then $$E[\hat \theta^2_1] = E[\hat \theta^2_2]$$

Even if this assumption is correct, my answer reduces down to $$ \theta = \theta$$ which doesn't really get me anywhere..

Where am I going wrong? What am I missing?


1 Answer 1


You are wrong in (**). If both first and second moments of estimates are equal, the variances should be equal too. But this is not the case. And also the argument in (*) is inconsistent: the expectation of sum is a sum of expectations whenever they exist. No indenedence is needed,

Next, I do not understand what are you doing firther and what for. You need to calculate the variance of $k_1\hat \theta_1 + k_2\hat \theta_2$ and then minimize it over any $k_1, k_2$ s.t. $k_1+k_2=1$.

Use the variance properties: for independent random variables with finite variances $$\text{Var}[X+Y]=\text{Var}[X]+\text{Var}[Y].$$ And the other property: if $c$ is a constant, $$\text{Var}[cX]=c^2\text{Var}[X].$$

Then $$ \text{Var}[k_1\hat \theta_1 + k_2\hat \theta_2]=k_1^2\text{Var}[\hat \theta_1] + k_2^2\text{Var}[\hat \theta_2] = k_1^2 \text{Var}[\hat \theta_1]+(1-k_1)^2 3\text{Var}[\hat \theta_1]. $$ Find $k_1$ that provides minimum of this expression and then find $k_2=1-k_1$.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .