4
$\begingroup$

I've been having trouble figuring out how to directly prove that the covariance of the multivariate Gaussian distribution $p(x) = \dfrac{1}{{{(2\pi)}^{\frac{n}{2}}}|\Sigma|^{\frac{1}{2}}}\exp\{{-\dfrac{1}{2}(x - \mu)^T \Sigma^{-1}(x-\mu)}\}$ where $x \in \mathbb{R}^n$ and $\Sigma$ is positive definite, is actually $\Sigma$.

That is to say, given $\Sigma$ and the above density function, I want to prove that the covariance is $\Sigma$. I've been able to prove that the mean is $\mu$ by diagonalizing the matrix $\Sigma$ and integrating, but I'm struggling to do it for the proof that the covariance is $\Sigma$. Can anyone help?

$\endgroup$
2
  • $\begingroup$ Do you know how to do it in the univariate case? $\endgroup$ Aug 28, 2016 at 6:29
  • $\begingroup$ yes. it's the multivariate that confuses me $\endgroup$
    – pyrrhic
    Aug 28, 2016 at 7:04

1 Answer 1

4
$\begingroup$

You need to perform a linear change of variables to do this. Let $H$ be a solution to $H^TH=\Sigma^{-1}$ (it exists because $\Sigma$ is positive definite). We will consider the random vector $y=H(x-\mu)$.

We have $y^Ty=(x-\mu)^TH^TH(x-\mu)=(x-\mu)^T\Sigma^{-1}(x-\mu)$ and $|H||\Sigma|^{1/2}=1$. So the pdf of $y$ is given by

$p(y)=\frac1{|H|}p(x)=\frac{1}{|H|}\frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}}\exp\left(-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)\right)= \frac{1}{(2\pi)^{n/2}}\exp\left(-\frac12y^Ty\right)$

This means that the entries of $y$ are independent standard normal variables, so $\mathrm{Cov}[y]=I$.

In general we have $\mathrm{Cov}[Ax+b]=A\mathrm{Cov}[X]A^T$ (this is the multivariate version of $\mathrm{Var}(ax+b)=a^2\mathrm{Var}(x)$, and can be shown straightforwardly by considering the definition $\mathrm{Cov}[X]=\mathbb{E}[XX^T]-\mathbb{E}[X]\mathbb{E}[X]^T$).

So in our case $I=\mathrm{Cov}[y]=\mathrm{Cov}[H(x-\mu)]=H\mathrm{Cov}[x]H^T$, and $\mathrm{Cov}[x]=H^{-1}(H^T)^{-1}=(H^TH)^{-1}=\Sigma$.

$\endgroup$
4
  • $\begingroup$ I have a relevant question though. Why is sigma positive definite? In general a covariance matrix needs only to be positive-semidefinite right? $\endgroup$
    – Zzy1130
    Mar 25, 2020 at 3:29
  • $\begingroup$ essentially my question is where does that inverse sigma come from $\endgroup$
    – Zzy1130
    Mar 25, 2020 at 4:15
  • 1
    $\begingroup$ @Zzy1130: That's simply an assumption stated explicitly in the question. If the covariance is not positive definite, the pdf is not given by the formula shown in the question (in fact there isn't a pdf in the usual sense), and we have an entirely different problem on our hands, which we can describe and solve separately. $\endgroup$ Mar 25, 2020 at 14:28
  • $\begingroup$ I see. Thank you so much. $\endgroup$
    – Zzy1130
    Mar 26, 2020 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.