Re-writing exponent of Multivariate Gaussian

In Bishop's Pattern Recognition and Machine Learning (ISBN-13: 978-0387-31073-2), Bishop writes on page 86:

This is an example of a rather common operation associated with Gaussian distributions, sometimes called ‘completing the square’, in which we are given a quadratic form defining the exponent terms in a Gaussian distribution, and we need to determine the corresponding mean and covariance. Such problems can be solved straightforwardly by noting that the exponent in a general Gaussian distribution $N(x|\mu,\Sigma)$ can be written: $$-\dfrac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu) = -\dfrac{1}{2}x^T \Sigma^{-1} x + x^T \Sigma^{-1} \mu + \text{const}$$

Where $\mu$ is the mean of a multivariate Gaussian and $\Sigma$ is the covariance matrix.

How was the above equation derived? Is there a name for this technique (Looking up completing the square doesn't yield anything)?

• Do understand the "complete the square" technique, do you know how to use it to solve the (scalar) quadratic equation $a x^2 + b x +c=0$ ? – leonbloy Jun 15 '15 at 14:22

The left hand side is the density of a multivariate Gaussian distribution.

To get to the right hand side, since this is linear algebra:

$$-\dfrac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu)$$ $$= -\dfrac{1}{2}x^T \Sigma^{-1} x +\dfrac{1}{2}\mu^T \Sigma^{-1} x +\dfrac{1}{2} x^T \Sigma^{-1} \mu - \dfrac{1}{2}\mu^T \Sigma^{-1} \mu$$

and the second and third terms are equal by the symmetry of the covariance matrix $\Sigma$ and so $\Sigma^{-1}$ while the fourth term is a constant.

So if you have a problem which is broadly like the right-hand side, you can transform it into the left-hand side and then use the known properties of a multivariate Gaussian. It is rather like the quadratic completing the square:

$$ax^2+bx+c = a\left(\left(x+\frac{b}{2a}\right)^2 - \left(\frac{\sqrt{b^2-4ac}}{2a}\right)^2\right)$$

• Haha, woops. Thanks. – lycus Jun 15 '15 at 14:27