if a matrix is sandwiched between an inverse covariance matrix and a covariance matrix, does it stay the same? Suppose we have a matrix $E$ that is a matrix of the eigenvectors of some covariance matrix $\Sigma$. Does 
$$\Sigma^{-1} E \Sigma =E \text{ ?} $$
I have tested it in Matlab and it seems the answer is no. But equation 6 in this blogpost seems to suggest so. I'm stuck and can't figure out how we finally get the last equation $$E^T E = W_D W_D^T$$
It seems to me that we can arrive at the last equation only if left multiplying $E$ by an inverse of $\Sigma$ and right multiplying by $\Sigma$ does not do anything to $E$. Please help. 
 A: The best way to understand that blog post is to just derive the whole thing from scratch.
We are given an arbitrary real "data matrix" $X$, and define $\Sigma$ to be $\frac1n X X^{\mathsf T}$. Because $\Sigma$ is a real symmetric matrix, the spectral theorem tells us that it has an orthonormal basis of eigenvectors: there is a matrix $E$ such that $E^{-1} = E^{\mathsf T}$ for which $E^{-1} \Sigma E$ is a diagonal matrix $D$.
The claim proven in the blog post is that if we define $Y = E^{\mathsf T}X$, then $\frac1n YY^{\mathsf T} = D$ (which can be interpreted as saying that the columns of $Y$ are uncorrelated).
To see this, just substitute in all the definitions:
\begin{align}
  \frac1n Y Y^{\mathsf T} &= \frac1n (E^{\mathsf T}X)(E^{\mathsf T}X)^{\mathsf T} \\
 &= \frac1n E^{\mathsf T} X X^{\mathsf T} E \\
 &= E^{\mathsf T} (\frac1n X X^{\mathsf T}) E \\
 &= E^{\mathsf T} \Sigma E \\
 &= E^{-1} \Sigma E = D.
\end{align}
Aside from some matrix multiplication mistakes, I am annoyed at the general structure of the proof: it assumes (for no reason) that the matrix $W_D$ exists, then proves that it's equal to $E^{\mathsf T}$ in a somewhat sketchy way. (As written, this is not even necessarily true: in degenerate cases, it's possible that there is more than one matrix $W_D$ that works!) It's better to just prove that $E^{\mathsf T}$ is the transformation that does we want, as above.
A: The author of that site messed up, but it seems fixable. It should say that $D=W_D\Sigma W_D$ in the derivation. Continuing from then on, you will see that $W_D=E^T$ satisfies (what should have been) the equation at the end. 
