1
$\begingroup$

Is $\Sigma = A^T\Sigma' A$ the same as $\Sigma = A\Sigma' A^T$ where $\Sigma$ is a symmetrical matrix and $\Sigma'$ is a diagonal matrix? If so, how can I prove this?

My textbook explains that a multivariate normal distribution can be written in terms of a diagonalised form of covariance $\Sigma$, using $\Sigma = A^T\Sigma' A$, by the process of singular value decomposition. However, other online sources explaining covariance decomposition mainly use $\Sigma = A^T\Sigma' A$ where it is described as an eigendecomposition. I'm confused as to whether they are the same thing.

$\endgroup$
  • $\begingroup$ The transpose is equal to itself, and so will be the diagonalised sigma. I can see why they're equal to each other now - thank you! $\endgroup$ – ru111 Oct 15 '17 at 20:36
  • $\begingroup$ In this case is eigendecomposition the same thing as SVD..? $\endgroup$ – ru111 Oct 15 '17 at 20:37
0
$\begingroup$

Their different in the sense that you can't get from $A^T\Sigma' A$ to $A \Sigma' A^T$ (i.e. using the same matrix, $A$, in both forms) but they are the same in the sense that if $B = A^T$ then $A^T\Sigma' A = B \Sigma' B^T$.

In general, if $\Sigma$ is a symmetric matrix, then we can choose a matrix $A$ consisting of eigenvectors of $\Sigma$ (as columns) such that $A^{-1} = A^T$ (i.e. $AA^T = I$) then to diagonalize $\Sigma$ we can write $\Sigma = A \Sigma' A^T$ where $\Sigma'$ is diagonal.

Alternatively, if we take $B = A^T$ then we have $B\Sigma B^T = \Sigma'$ (swapping the roles of $\Sigma$ and $\Sigma'$) and so $\Sigma = B^T \Sigma' B$ instead. Here the matrix $B$ has eigenvectors of $\Sigma$ as rows.

$\endgroup$
1
$\begingroup$

Those forms are equivalent. The first matrix in the product of three is a transpose of the third - it is up to you to decide which one you will call $A$ and which one you'll call $A^T$.

$\endgroup$
0
$\begingroup$

While it is true that the forms are equivalent (just take $A$ in place of $A^\top$) they are not equal which is probably what you are confused about.
To see that they are not equal just take some arbitrary matrices and multiply out, for example:

$$ A = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} \qquad \Sigma' = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix} $$ then $$ A \Sigma' A^{\top} = \begin{pmatrix} 2 & 0 \\ 0 & 0 \end{pmatrix} $$ but $$ A^{\top} \Sigma' A = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix} $$

$\endgroup$
  • $\begingroup$ Do you mean you can write $\Sigma$ in terms of both, but the diagonal matrix and/or A will be different? $\endgroup$ – ru111 Oct 15 '17 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.