4
$\begingroup$

This may be trivial.

While covariance matrices of random variables are positive semi-definite, does the converse hold true as well, that positive semi-definite matrices are also valid covariance matrices of random variables?

Wikipedia says this is the case, however, I don't follow the argument:

... the covariance matrix of a multivariate probability distribution is always positive semi-definite. (...) Conversely, every positive semi-definite matrix is the covariance matrix of some multivariate distribution.

Grateful for any explanations.

$\endgroup$
3
$\begingroup$

If $X$ is a multivariate distribution (dimension N), and if $A$ is a positive semidefinite $N\times N$ matrix, then $Y=AX$ has covariance matrix cov($Y$) related to the covariance matrix cov($X$) of $X$ by $$ \mbox{cov}(Y)=A\;\mbox{cov}(X)\,A^{T}. $$ So if you start with independent components of $X$ so that cov($X$)=I, then $$ \mbox{cov}(Y)=AA^{T}. $$ Then, by arguing that any positive semidefinite matrix $M$ can be written as $AA^{T}$, you end up with $Y$ whose covariance matrix is $M$. In fact, you can write $M=A^{2}$ with $A=A^{T}$, which isn't too hard to show by choosing an orthonormal basis of eigenvectors for $M$ (which is one form of the spectral theorem.)

$\endgroup$
  • $\begingroup$ That's a great way to think about it. Thanks for the explanation. $\endgroup$ – wfh Feb 9 '14 at 0:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.