# How to represent EigenValues of covariance matrix

If I have a random mean zero vector X and a covariance matrix $\Sigma = E(XX^T)$, the eigen values of $\Sigma$ are given as $\lambda_1..._d$, how would I represent the covariance matrix in terms of its eigenvalues and eigenvectors? Also what is the relation between the covariance matrix and variance?

## 1 Answer

The covariance matrix is symmetric, so has an orthonormal eigenvector basis by the spectral theorem. To recover $\Sigma$ from normalised column-eigenvectors and eigen-values, just form a matrix $P$ by writing your eigenvectors in order and a diagonal matrix D with the eigenvalues along the diagonal, in the corresponding order to P. Since P is orthonormal, $P^{-1}=P^t$ and $\Sigma = P^{-1}DP$. Also see https://en.wikipedia.org/wiki/Matrix_decomposition#Eigendecomposition. The only statistical fact used here is that $\Sigma^t=\Sigma$, everything else is general linear algebra.

Re terminology: Some authors call $\Sigma$ the variance because it is the reasonable generalisation of uni-variate variance to multivariate distributions: The variance of each component of your probability Vector sits on the diagonal of $\Sigma$, while covariances are off-diagonal - in the univariate case the 1-by-1 "Covariance matrix" is trivially diagonal, so just equal to the variance. (See "Conflicting nomenclatures and notations" on https://en.wikipedia.org/wiki/Covariance_matrix)