I've been having trouble figuring out how to directly prove that the covariance of the multivariate Gaussian distribution $p(x) = \dfrac{1}{{{(2\pi)}^{\frac{n}{2}}}|\Sigma|^{\frac{1}{2}}}\exp\{{-\dfrac{1}{2}(x - \mu)^T \Sigma^{-1}(x-\mu)}\}$ where $x \in \mathbb{R}^n$ and $\Sigma$ is positive definite, is actually $\Sigma$.

That is to say, given $\Sigma$ and the above density function, I want to prove that the covariance is $\Sigma$. I've been able to prove that the mean is $\mu$ by diagonalizing the matrix $\Sigma$ and integrating, but I'm struggling to do it for the proof that the covariance is $\Sigma$. Can anyone help?

  • $\begingroup$ Do you know how to do it in the univariate case? $\endgroup$ – Daniel McLaury Aug 28 '16 at 6:29
  • $\begingroup$ yes. it's the multivariate that confuses me $\endgroup$ – pyrrhic Aug 28 '16 at 7:04

You need to perform a linear change of variables to do this. Let $H$ be a solution to $H^TH=\Sigma^{-1}$ (it exists because $\Sigma$ is positive definite). We will consider the random vector $y=H(x-\mu)$.

We have $y^Ty=(x-\mu)^TH^TH(x-\mu)=(x-\mu)^T\Sigma^{-1}(x-\mu)$ and $|H||\Sigma|^{1/2}=1$. So the pdf of $y$ is given by

$p(y)=\frac1{|H|}p(x)=\frac{1}{|H|}\frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}}\exp\left(-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)\right)= \frac{1}{(2\pi)^{n/2}}\exp\left(-\frac12y^Ty\right)$

This means that the entries of $y$ are independent standard normal variables, so $\mathrm{Cov}[y]=I$.

In general we have $\mathrm{Cov}[Ax+b]=A\mathrm{Cov}[X]A^T$ (this is the multivariate version of $\mathrm{Var}(ax+b)=a^2\mathrm{Var}(x)$, and can be shown straightforwardly by considering the definition $\mathrm{Cov}[X]=\mathbb{E}[XX^T]-\mathbb{E}[X]\mathbb{E}[X]^T$).

So in our case $I=\mathrm{Cov}[y]=\mathrm{Cov}[H(x-\mu)]=H\mathrm{Cov}[x]H^T$, and $\mathrm{Cov}[x]=H^{-1}(H^T)^{-1}=(H^TH)^{-1}=\Sigma$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.