I am trying to catch up with my statistics. I want to understand what are multivariate Gaussian or multivariate normal distribution. I understand what a univariate Gaussian is any variable $X$ such that $X\sim N(\mu,\sigma^2),\mu\in\mathbb R, \sigma^2>0$ (except for complex numbers, isn't something squared always greater than $0$ ?), that is to say, if $x_i\in X\sim N(0,\sigma^2)$ There is lots of chance that it pertains to something that looks like this :

univariate Gaussian

And as far as

if $x_i\in X\sim N(\mu,\sigma^2)$ with $\mu =\begin{bmatrix}0\\0\end{bmatrix}$ and $\sigma =\begin{bmatrix}1 & 3/5\\3/5 & 2\end{bmatrix}$ (I don't understand the covariance matrix, especially the bottom right corner) It would have be part of something like this :

enter image description here

But then I don't understand its definition given on ... which I found back on Wikipedia :

A random vector $X = (X_1, …, X_k)'$ is said to have the multivariate normal distribution if it satisfies the following equivalent conditions.

  1. Every linear combination of its components $Y = a_1X_1 + … + a_kX_k$ is normally distributed i.e. $a^TX=\sum_{i=1}^na_iX_i$ is Gaussian $\forall a\in\mathbb R^n$ (what $a$ has to do there ?)

Would it mean, if we had three random variables, that combining two of them would be a Gaussian, it would look like the picture above. Yet, it isn't a Gaussian, it's a bi dimensional Gaussian !

  • 1
    $\begingroup$ "except for complex numbers, isn't something squared always greater than 0 ?" Not if it's zero. What that condition is telling you is that $\sigma^2$ is not allowed to be zero. $\endgroup$ – Rahul Dec 26 '17 at 16:27

No, the property means that if every projection from $k$ dimensions to $1$ (i.e. any linear combination) yields a normally distributed variable, then the $k$D variable is a multivariate Gaussian. [A single projection is insufficient. On your second plot, you see two different projections giving $1$D Gaussians. If I am right, $k$ independent projections are sufficient.]

If you want to picture a multivariate Gaussian, think of the locus of equiprobable points. In $1$D, you get two isolated points. In $2$, an ellipse. In $k$D, the generalization of an ellipse, i.e. an hyperellipsoid.

  • $\begingroup$ @MichaelHardy: that's right. $\endgroup$ – Yves Daoust Dec 26 '17 at 16:58
  • $\begingroup$ Thank you for your answer ! I understand everything but the bracket point ! $\endgroup$ – IggyPass Dec 27 '17 at 0:23
  • $\begingroup$ @Marine1: this is a little technicality, don't worry. $\endgroup$ – Yves Daoust Dec 27 '17 at 9:21

If you have three random variables $X_1,X_2,X_3$ then the statement that the triple $(X_1,X_2,X_3)$ has a multivariate Gaussian distribution means that regardless of which three numbers $a_1,a_2,a_3$ you choose, the linear combination $a_1X_1 + a_2X_2 + a_3X_3$ has a univariate Gaussian distribution. For example, $40X_1 + 5X_2 - 13X_3$ has a univariate Gaussian distribution, and similarly for any other triple of numbers besides $(40,5,-13).$

  • $\begingroup$ I understand ! So why should we bother with multivariate distribution if for any combination of the variables which are part of a multivariate distribution we have a univariate distribution ? $\endgroup$ – IggyPass Dec 27 '17 at 0:02
  • $\begingroup$ @Marine1 : For one thing, how would you find confidence ellipsoids? And how would you talk about the conditional distribution of $X_1,X_2$ given $X_3, X_4\text{?}$ And there's the multivariate counterpart of the chi-square distribution, the (matrix-valued) Wishart distribution, used for a number of things in statistics. $\endgroup$ – Michael Hardy Dec 27 '17 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.