Confusion about the expected distance between two gaussian random variables This math stack exchange answer gives a very nice solution for the expected squared L2 norm of any gaussian random variable:
$$ \mathbb{E}\{\|Z\|^2\} = \| \mu \|^2 + Tr(\Sigma)$$
I walked through the derivation and it all makes sense to me, but I'm having trouble lining this up with my intuition about the gaussian distribution. As I understand, we can visualize the covariance $\Sigma$ as an ellipsoid, where the major/minor axes are given by the eigenvectors of $\Sigma$, and the eigenvalues tell us the scale of the ellipsoid in that direction. Also, the i'th diagonal element of $\Sigma$ is the variance of $Z_i$.
So $Tr(\Sigma)$ is the sum of the variances of $Z_i$. It makes sense that this should be the case. However, we also should be able to rotate the gaussian distribution about the mean and still have the same value for $\mathbb{E}\{\|Z\|^2\}$. But doing so would involve changing $Tr(\Sigma)$, right?
To concisely summarize: Why is $Tr(\Sigma)$ invariant to rotations about the mean?
 A: The question has been answered, but I would like to point out that your identity has nothing to do with the Gaussian distribution and it would hold for any random vector $Z$; rather it is a simple consequence of the bias-variance decomposition of the mean squared error:
$$E[\|\hat\theta-\theta\|^2]=\|E[\hat\theta]-\theta\|^2+ tr(V(\hat\theta)),$$
where $\hat \theta$ is some random vector thought of as an estimator for parameter $\theta$. Your identity is the case where $\hat\theta=Z\sim N(\mu,\Sigma),\theta=0.$

Here is a simple proof to convince you. First, recall some properties:

*

*$V(X)=E[XX']-E[X]E[X']$

*$tr(A+B)=tr(A)+tr(B)$

*$tr(AB)=tr(BA)$.

Then we have
$$\begin{align}tr(V(\hat\theta))&=tr(V(\hat\theta-\theta))\\
&=tr(E[(\hat\theta-\theta)(\hat\theta-\theta)']-E[\hat\theta-\theta]E[(\hat\theta-\theta)'])\quad (1)\\
&=tr(E[(\hat\theta-\theta)(\hat\theta-\theta)'])-tr(E[\hat\theta-\theta]E[(\hat\theta-\theta)'])\quad (2)\\
&=tr(E[(\hat\theta-\theta)'(\hat\theta-\theta)])-tr(E[\hat\theta-\theta]'E[\hat\theta-\theta])\quad (3)\\
&=E[\|\hat\theta-\theta\|^2]-\|E[\hat\theta]-\theta\|^2\\
\implies E[\|\hat\theta-\theta\|^2]&=\|E[\hat\theta]-\theta\|^2+ tr(V(\hat\theta))& \end{align}$$
A: Your rotation would entail replacing $\Sigma$ with $U \Sigma U^\top$ for some rotation $U$. Then $\operatorname{Tr}(U \Sigma U^\top) = \operatorname{Tr}(\Sigma U^\top U) = \operatorname{Tr}(\Sigma)$.
