6
$\begingroup$

One can easily prove that every covariance matrix is positive semi-definite. I come across many claims that the converse is also true; that is,

Every symmetric positive semi-definite matrix is a covariance marix of some multivariate distribution.

Is it true? If it is, how can we prove it?

$\endgroup$
5
$\begingroup$

The answer is affirmative. Every positive semidefinite matrix $C$ can be orthogonally diagonalised as $QD^2Q^T$, where $Q$ is a real orthogonal matrix and $D$ is a nonnegative diagonal matrix. Let $\mathbf{Z}$ be a random vector following the standard multivariate normal distribution $N(0,I_n)$. It is straightforward to verify that $C$ is the covariance matrix of $\mathbf{X}=QD\mathbf{Z}$.

$\endgroup$
  • $\begingroup$ No need to be normal, though. $\endgroup$ – leonbloy Oct 9 '14 at 13:25
1
$\begingroup$

The wikipedia article on covariance matrices answers that (the excerpt below is taken verbatim from that article):

From the identity just above, let $\mathbf{b}$ be a $(p \times 1)$ real-valued vector, then: $$\operatorname{var}(\mathbf{b}^{\rm T}\mathbf{X}) = \mathbf{b}^{\rm T} \operatorname{var}(\mathbf{X}) \mathbf{b},$$ which must always be nonnegative since it is the variance of a real-valued random variable and the symmetry of the covariance matrix's definition it follows that only a positive-semidefinite matrix can be a covariance matrix. The answer to the converse question, whether every symmetric positive semi-definite matrix is a covariance matrix, is "yes". To see this, suppose $\mathbf{M}$ is a $p\times p$ positive-semidefinite matrix. From the finite-dimensional case of the spectral theorem, it follows that $\mathbf{M}$ has a nonnegative symmetric square root, that can be denoted by $\mathbf{M}^{1/2}$. Let $\mathbf{X}$ be any $p\times 1$ column vector-valued random variable whose covariance matrix is the $p\times p$ identity matrix. Then: $$\operatorname{var}(\mathbf{M}^{1/2}\mathbf{X}) = \mathbf{M}^{1/2} (\operatorname{var}(\mathbf{X})) \mathbf{M}^{1/2} = \mathbf{M}.$$

$\endgroup$
  • 1
    $\begingroup$ While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. $\endgroup$ – Nick Peterson Sep 10 '13 at 17:11
  • $\begingroup$ I provided the article section. Sorry, its the first question that I answer on this website :-) $\endgroup$ – DCG Sep 10 '13 at 17:25
  • $\begingroup$ I made some minor formatting changes, as well as the major formatting change of putting the quoted part in blockquotes. Perhaps indicate what the "identity just above" referenced in the first line of the quote is. $\endgroup$ – user642796 Sep 10 '13 at 17:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.