# Deriving the Covariance of Multivariate Gaussian

I've been having trouble figuring out how to directly prove that the covariance of the multivariate Gaussian distribution $p(x) = \dfrac{1}{{{(2\pi)}^{\frac{n}{2}}}|\Sigma|^{\frac{1}{2}}}\exp\{{-\dfrac{1}{2}(x - \mu)^T \Sigma^{-1}(x-\mu)}\}$ where $x \in \mathbb{R}^n$ and $\Sigma$ is positive definite, is actually $\Sigma$.

That is to say, given $\Sigma$ and the above density function, I want to prove that the covariance is $\Sigma$. I've been able to prove that the mean is $\mu$ by diagonalizing the matrix $\Sigma$ and integrating, but I'm struggling to do it for the proof that the covariance is $\Sigma$. Can anyone help?

• Do you know how to do it in the univariate case? Aug 28, 2016 at 6:29
• yes. it's the multivariate that confuses me Aug 28, 2016 at 7:04

You need to perform a linear change of variables to do this. Let $H$ be a solution to $H^TH=\Sigma^{-1}$ (it exists because $\Sigma$ is positive definite). We will consider the random vector $y=H(x-\mu)$.

We have $y^Ty=(x-\mu)^TH^TH(x-\mu)=(x-\mu)^T\Sigma^{-1}(x-\mu)$ and $|H||\Sigma|^{1/2}=1$. So the pdf of $y$ is given by

$p(y)=\frac1{|H|}p(x)=\frac{1}{|H|}\frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}}\exp\left(-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)\right)= \frac{1}{(2\pi)^{n/2}}\exp\left(-\frac12y^Ty\right)$

This means that the entries of $y$ are independent standard normal variables, so $\mathrm{Cov}[y]=I$.

In general we have $\mathrm{Cov}[Ax+b]=A\mathrm{Cov}[X]A^T$ (this is the multivariate version of $\mathrm{Var}(ax+b)=a^2\mathrm{Var}(x)$, and can be shown straightforwardly by considering the definition $\mathrm{Cov}[X]=\mathbb{E}[XX^T]-\mathbb{E}[X]\mathbb{E}[X]^T$).

So in our case $I=\mathrm{Cov}[y]=\mathrm{Cov}[H(x-\mu)]=H\mathrm{Cov}[x]H^T$, and $\mathrm{Cov}[x]=H^{-1}(H^T)^{-1}=(H^TH)^{-1}=\Sigma$.

• I have a relevant question though. Why is sigma positive definite? In general a covariance matrix needs only to be positive-semidefinite right? Mar 25, 2020 at 3:29
• essentially my question is where does that inverse sigma come from Mar 25, 2020 at 4:15
• @Zzy1130: That's simply an assumption stated explicitly in the question. If the covariance is not positive definite, the pdf is not given by the formula shown in the question (in fact there isn't a pdf in the usual sense), and we have an entirely different problem on our hands, which we can describe and solve separately. Mar 25, 2020 at 14:28
• I see. Thank you so much. Mar 26, 2020 at 9:35