In my textbook it is said:
... the mean square error for any estimator is equal to its variance plus the square of its bias.
I'm having trouble understanding how do this translate in practice. I went looking for problem questions that tackles this subject.
Here is one,
Consider two estimators $d_1$ and $d_2$ of a parameter $\theta$. If $E[d_1] = \theta$, $var(d_1) = 6$ and $E[d_2] = \theta + 2$, $var(d_2) = 2$, which estimators should be preffered?
The solution for this problem revolves arround computing the Mean Square Error (MSE) for both estimators and then deciding which to pick. Looking up the solution we have this:
Since $d_1$ is an unbiased estimator its MSE is equal to its variance. For $d_2$ the MSE is (variance + square of its bias):
Note: the formula for the $ MSE = r(d_i, \theta) = E[(d_i - \theta)^2] $. Also when you see $E(...)^2$ it means $E[(...)^2]$.
I don't understand where and how the square of the bias is being computed?
Why do we do $+ 2$ after substituing $\theta$ with $\theta + 2$ in the second line?
Where does the part $4E[d_2 - (\theta + 2)] + 4$ comes from? (Is it our square of the bias?)
Why does $E[d_2 - (\theta + 2)] = 0$ ?
Why does $E[(d_2 - (\theta + 2))^2] = var(d_2) = 2$ ? I thought we had a biased estimator?