# Choosing constant to minimize mean square error

Can you please help me show the statement below? I am not exactly sure where to start.

-

## 2 Answers

As a function of $c$, the quadratic $\mathrm E(S^4)c^2-2\sigma^2\mathrm E(S^2)c+\sigma^4$ is minimum at $c=\sigma^2\mathrm E(S^2)/\mathrm E(S^4)$. To compute this ratio, the simplest approach could be to compute $\mathrm E(S^2)$ and $\mathrm E(S^4)$.

For starters, would you know how to compute $\mathrm E(S^2)$? Hint: the value of $\mathrm E(S^2)$ does not depend on the sample being drawn from a normal distribution, but only on the parameters $\mu$ and $\sigma^2$.

-
Isn't E(S^2) just the same as sigma^2 since it is unbiased? How would E(S^4) be found? –  icobes Oct 15 '11 at 6:46
(1) What is the definition of $S^2$ you want to use? (2) Answering that $E(S^2)=\sigma^2$ because $S^2$ is [an] unbiased [estimator of the variance], is akin to playing with words and this is not what I suggested. My suggestion was rather to reprove this comparatively simple fact (or to remember how its proof goes), in order to be ready to tackle $E(S^4)$. In the end, this is up to you: either you wait for a complete answer to appear on this page (something which may very well happen, unfortunately), or you start doing maths yourself... –  Did Oct 15 '11 at 6:56
Would E(S^2) be sigma^2 and E(S^4) be mew^2 + sigma^2? I calculated these explicitly and those were the values I got. –  icobes Oct 16 '11 at 0:43
(1) (bis) What is the definition of $S^2$ you want to use? There are several options and as long as you have not said which one you use, the discussion is pointless. (3) For homogeneity reasons, $E(S^4)$ should be of degree $4$ in the variables $\mu$ and $\sigma$ hence the formula for $E(S^4)$ in your last comment cannot hold, whatever option you choose to define $S^2$. (4) NEVERTHELESS, I suggest you write here (either in your post or in an answer) how you got $\mu^2+\sigma^2$ for $E(S^4)$ and $\sigma^2$ for $E(S^2)$, so that we have something concrete to discuss. –  Did Oct 16 '11 at 7:50
I just saw this post which tremendously helped and gave me a good method for finding E(S^4): math.stackexchange.com/questions/72975/… –  icobes Oct 16 '11 at 16:34

@icobes Hint for finding $E(S^2)$: Using notation that the parameters $\sigma^2=Var(S)$ and $\mu = E(S)$, remember the definition $Var(S) = E((S-\mu)^2)$. That gives $Var(S) = E(S^2)-(E(S))^2$ by linearity (why?).

To finish finding $E(S^2)$, how can you find $E(S^2)$ in terms of $\sigma^2$ and $\mu$?

Moving on, can we use a similar method to find $E(S^4)$ and how does the normal distribution start to come in?

-
Avi: hmmm... there is no formula for $S$ except that $S$ is the square root of $S^2$ hence for example $E(S)=\mu$ is not true (this $S$ is not the sample sum nor the sample mean). –  Did Oct 15 '11 at 7:18
@DidierPiau Oops, you are correct, I misinterpreted the question. Is it appropriate etiquette for me to edit or delete my previous comment to avoid confusion? –  Avi S. Oct 18 '11 at 23:05