# Covariance matrix decomposition - is $A^T\Sigma' A = A\Sigma' A^T$?

Is $\Sigma = A^T\Sigma' A$ the same as $\Sigma = A\Sigma' A^T$ where $\Sigma$ is a symmetrical matrix and $\Sigma'$ is a diagonal matrix? If so, how can I prove this?

My textbook explains that a multivariate normal distribution can be written in terms of a diagonalised form of covariance $\Sigma$, using $\Sigma = A^T\Sigma' A$, by the process of singular value decomposition. However, other online sources explaining covariance decomposition mainly use $\Sigma = A^T\Sigma' A$ where it is described as an eigendecomposition. I'm confused as to whether they are the same thing.

• The transpose is equal to itself, and so will be the diagonalised sigma. I can see why they're equal to each other now - thank you! – ru111 Oct 15 '17 at 20:36
• In this case is eigendecomposition the same thing as SVD..? – ru111 Oct 15 '17 at 20:37

## 3 Answers

Their different in the sense that you can't get from $A^T\Sigma' A$ to $A \Sigma' A^T$ (i.e. using the same matrix, $A$, in both forms) but they are the same in the sense that if $B = A^T$ then $A^T\Sigma' A = B \Sigma' B^T$.

In general, if $\Sigma$ is a symmetric matrix, then we can choose a matrix $A$ consisting of eigenvectors of $\Sigma$ (as columns) such that $A^{-1} = A^T$ (i.e. $AA^T = I$) then to diagonalize $\Sigma$ we can write $\Sigma = A \Sigma' A^T$ where $\Sigma'$ is diagonal.

Alternatively, if we take $B = A^T$ then we have $B\Sigma B^T = \Sigma'$ (swapping the roles of $\Sigma$ and $\Sigma'$) and so $\Sigma = B^T \Sigma' B$ instead. Here the matrix $B$ has eigenvectors of $\Sigma$ as rows.

Those forms are equivalent. The first matrix in the product of three is a transpose of the third - it is up to you to decide which one you will call $A$ and which one you'll call $A^T$.

While it is true that the forms are equivalent (just take $A$ in place of $A^\top$) they are not equal which is probably what you are confused about.
To see that they are not equal just take some arbitrary matrices and multiply out, for example:

$$A = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} \qquad \Sigma' = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}$$ then $$A \Sigma' A^{\top} = \begin{pmatrix} 2 & 0 \\ 0 & 0 \end{pmatrix}$$ but $$A^{\top} \Sigma' A = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}$$

• Do you mean you can write $\Sigma$ in terms of both, but the diagonal matrix and/or A will be different? – ru111 Oct 15 '17 at 20:42