Mean square law of large numbers I have the following line in my notes (which I believe is flawed):
$$E\left[ \left(\hat{\mu}(N)-\mu\right)^2 \right] =E\left[ \left( \frac{1}{N} \sum_{i=1}^N(x_i-\mu) \right)^2 \right] =\frac{\sigma^2}{N}$$
I think the error is in brackets, $\frac{1}{N}$ us not multiplied by $\sum_{i=1}^N(x_i-\mu)$, but only: $\sum_{i=1}^Nx_i$, since $\hat{\mu}$ is defined as the sample mean: $\frac{1}{N}\sum_{i=1}^N x_i$.
But even given that (and supposing that I am correct), I cannot arrive at the required result of $\sigma^2 / N$.
This is what I get:
$$E\left[ \left(\frac{1}{N}\sum_{i=1}^Nx_i-\mu\right)^2 \right] =\frac{1}{N^2}E\left[ \sum_{i=1}^N x_i^2 \right] -\frac{2\mu}{N}E\left[ \sum_{i=1}^N x_i \right] +\mu^2$$
EDIT:
If I have i.i.d RV's with finite mean, then can I develop the above by saying the following:
$E[X]=<X>=\frac{1}{N}\sum_{i=1}^Nx_i$ and $E[X^2]=<X^2>=\frac{1}{N}\sum_{i=1}^Nx_i^2$, therefore I have
$$E\left[ \left(\frac{1}{N}\sum_{i=1}^Nx_i-\mu\right)^2 \right] =\frac{1}{N^2}E\left[ \sum_{i=1}^N x_i^2 \right] -\frac{2\mu}{N}E\left[ \sum_{i=1}^N x_i \right] +\mu^2=\frac{1}{N}<X^2>-<X>^2=\frac{1}{N}(<X^2>-<X>^2)=\frac{\sigma^2}{N}$$
 A: The trick is a clever grouping of terms:
\begin{align}E \left [ \left (  \frac{1}{N} \sum_{i=1}^N (x_i) - \mu \right )^2 \right ] 
& = E \left [ \left ( \frac{1}{N} \sum_{i=1}^N (x_i-\mu) \right )^2 \right ] \\[5pt]
&= \frac{1}{N^2} E \left [ \sum_{i=1}^n (x_i-\mu)^2 + \sum_{i=1}^N \sum_{j=1,\;j \neq i}^N (x_i-\mu)(x_j-\mu) \right ] \\[5pt]
&= \frac{\sigma^2}{N} + \frac{1}{N^2} E \left [ \sum_{i=1}^N \sum_{j=1,\;j \neq i}^N (x_i-\mu)(x_j-\mu) \right ]\end{align}
Now, why is the last term zero?
What we have really shown here is two basic facts:


*

*The variance of a sum of uncorrelated variables is the sum of the variances.

*The variance of $\alpha X$ is $\alpha^2 \text{Var}(X)$, if $\alpha \in \mathbb{R}$. 


This technique can also be used to prove that the expected value of $\sum_{i=1}^N \left(x_i - \hat{\mu}(N)\right)^2$ is $(N-1)\sigma^2$ (which explains the somewhat mysterious $N-1$ in the standard formula for the sample variance).
A: The secret to the middle expression is simple:
if you add together $N$ copies of the exact same quantity,
you get $N$ times the original quantity.
$$ \sum_{i=1}^N \mu = N \mu.$$
So if we take $\frac1N$ of the sum, we get back the original quantity:
$$\frac1N \sum_{i=1}^N \mu = \frac1N ( N \mu) = \mu.$$
Now combine this with the already-known formula for $\hat\mu(N)$:
\begin{align}
\hat\mu(N) &= \frac1N \sum_{i=1}^N x_i, \\
\hat\mu(N) - \mu &= \frac1N \sum_{i=1}^N x_i - \mu  \\
 &= \frac1N \sum_{i=1}^N x_i - \frac1N \sum_{i=1}^N \mu  \\
 &= \frac1N \left(\sum_{i=1}^N x_i - \sum_{i=1}^N \mu \right). \\
\end{align}
Now apply the well-known fact that 
$\sum_{i=1}^N a_n - \sum_{i=1}^N b_n = \sum_{i=1}^N (a_n - b_n)$.
That is, instead of adding up all the $x_i$s and then subtracting off
the sum of all the $\mu$s in that order,
pair off each $x_i$ with one of the $\mu$s that we are going to subtract:
$$ \sum_{i=1}^N x_i - \sum_{i=1}^N \mu = \sum_{i=1}^N (x_i - \mu). $$
Therefore
$$ \hat\mu(N) - \mu 
= \frac1N \left(\sum_{i=1}^N x_i - \sum_{i=1}^N \mu \right)
 = \frac1N \sum_{i=1}^N (x_i - \mu). $$
The first equals sign in your question is simply taking the
expectation of the square of the quantity on both sides of an equation.
This is a somewhat long-winded way of showing how the first
equals sign in Ian's answer works.
Follow that answer for the rest of the derivation.
