0
$\begingroup$

In my textbook it is said:

... the mean square error for any estimator is equal to its variance plus the square of its bias.

I'm having trouble understanding how do this translate in practice. I went looking for problem questions that tackles this subject.

Here is one,

Consider two estimators $d_1$ and $d_2$ of a parameter $\theta$. If $E[d_1] = \theta$, $var(d_1) = 6$ and $E[d_2] = \theta + 2$, $var(d_2) = 2$, which estimators should be preffered?

The solution for this problem revolves arround computing the Mean Square Error (MSE) for both estimators and then deciding which to pick. Looking up the solution we have this:

Since $d_1$ is an unbiased estimator its MSE is equal to its variance. For $d_2$ the MSE is (variance + square of its bias): enter image description here

Note: the formula for the $ MSE = r(d_i, \theta) = E[(d_i - \theta)^2] $. Also when you see $E(...)^2$ it means $E[(...)^2]$.

I don't understand where and how the square of the bias is being computed?

Why do we do $+ 2$ after substituing $\theta$ with $\theta + 2$ in the second line?

Where does the part $4E[d_2 - (\theta + 2)] + 4$ comes from? (Is it our square of the bias?)

Why does $E[d_2 - (\theta + 2)] = 0$ ?

Why does $E[(d_2 - (\theta + 2))^2] = var(d_2) = 2$ ? I thought we had a biased estimator?

$\endgroup$

1 Answer 1

1
$\begingroup$

The point of writing $$d_2-\theta=\color{blue}{d_2-(\theta+2)}+2$$ is to compare $d_2$ to its expected value. Note that the blue expression has zero expectation, and the expectation of its square is just the variance of $d_2$ (by definition of variance).

You can also expand $E[(d_2-\theta)^2]$ without rewriting the expression in this way; you get $$E[d_2^2]-2\theta E[d_2]+\theta^2$$

But the first term is just the variance plus $E[d_2]^2$, which is $(\theta+2)^2$, and the second term is $-2\theta(\theta+2)$. Thus we have

$$2+(\theta+2)^2-2\theta^2-4\theta+\theta^2$$

Simplifying gives the same MSE we obtained the other way, $6$. The first solution is algebraically cleverer but by no means necessary.

$\endgroup$
4
  • $\begingroup$ Could you make it more obvious why $d_2 - (\theta + 2)$ has an expectation of 0? I have trouble seeing it. $\endgroup$
    – WindBreeze
    Commented Nov 16, 2019 at 18:48
  • $\begingroup$ Come on. You’re told $E[d_2]=\theta+2$. Now use the linearity of expectation applied to $d_2-(\theta+2)$. $\endgroup$ Commented Nov 16, 2019 at 18:51
  • $\begingroup$ But this is $d_2 - E[d_2]$ not $E[d_2] - E[d_2] = 0$, it's different, no? $\endgroup$
    – WindBreeze
    Commented Nov 16, 2019 at 19:28
  • $\begingroup$ I have no idea what you’re doing. $$E[d_2-(\theta+2)]=E[d_2]-E[\theta+2]=\theta+2-(\theta+2)=0$$ $\endgroup$ Commented Nov 16, 2019 at 20:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .