0
$\begingroup$

If I have a random mean zero vector X and a covariance matrix $\Sigma = E(XX^T)$, the eigen values of $\Sigma$ are given as $\lambda_1..._d$, how would I represent the covariance matrix in terms of its eigenvalues and eigenvectors? Also what is the relation between the covariance matrix and variance?

$\endgroup$
0
$\begingroup$

The covariance matrix is symmetric, so has an orthonormal eigenvector basis by the spectral theorem. To recover $\Sigma$ from normalised column-eigenvectors and eigen-values, just form a matrix $P$ by writing your eigenvectors in order and a diagonal matrix D with the eigenvalues along the diagonal, in the corresponding order to P. Since P is orthonormal, $P^{-1}=P^t$ and $\Sigma = P^{-1}DP$. Also see https://en.wikipedia.org/wiki/Matrix_decomposition#Eigendecomposition. The only statistical fact used here is that $\Sigma^t=\Sigma$, everything else is general linear algebra.

Re terminology: Some authors call $\Sigma$ the variance because it is the reasonable generalisation of uni-variate variance to multivariate distributions: The variance of each component of your probability Vector sits on the diagonal of $\Sigma$, while covariances are off-diagonal - in the univariate case the 1-by-1 "Covariance matrix" is trivially diagonal, so just equal to the variance. (See "Conflicting nomenclatures and notations" on https://en.wikipedia.org/wiki/Covariance_matrix)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.