Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How does one go from

$e^{(-\frac{1}{2\sigma_n^2}(\vec{y} - X^T\vec{w})^T(\vec{y}-X^T\vec{w}))}e^{-\frac{1}{2}\vec{w}\Sigma^{-1}\vec{w}}$


$e^{-\frac{1}{2}(\vec{w}- w')^T(\frac{1}{\sigma^2_n}XX^T + \Sigma^{-1})(\vec{w} - w')}$

where $w' = \sigma_n^{-2}(\sigma_n^{-2}XX^T + \Sigma^{-1})^{-1}$

I'm sure the algebra works out upon expansion, but how do you come up with this?

share|cite|improve this question

Arise from the grave, oh two year old question!

I was wondering about the same thing, and many people will wonder, as this expression comes up when you calculate posterior in Gaussian process with first exponent being likelihood and the second being prior on the weights. I found this expression in p. 9 of the wonderful book Gaussian Processes for Machine Learning by Rasmussen and Williams and was annoyed that I can't see how to complete squares in multivariate case. If someone could give some intuition, I would be very happy!

Meanwhile, for this answer to be of any use, let me show you a trick I came up with: from the Gaussian prior and Gaussian likelihood we expect Gaussian posterior. Gaussian posterior will have the form $$ e^{-\frac{1}{2}(w - \bar w)^T\Sigma\ (w-\bar w)} $$ with $\bar w$ beeing the mean and $\Sigma$ the covariance matrix. In order to find the mean, we take the derivative of the exponent in the question (there is one transposition missing in victor's post) $$ \left(\frac{1}{\sigma_n^2} (y - X^Tw)^T(y-X^Tw) + w^T\Sigma^{-1}w\right)'_w = \frac{-2}{\sigma_n^2}(y-X^Tw)^TX^T + 2w^T\Sigma^{-1}, $$ and equate it to zero. Solving for $w$ we recover the $\bar w = \sigma_n^{-2}(\sigma_n^{-2}XX^T + \Sigma^{-1})^{-1}$ part.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.