I'm reading through these notes that claim that if we expand the log of the multivariate gaussian distribution we get the following:

$$log(N(x_n|\mu_k, \Sigma_k)) = -\frac{1}{2} (x_n - \mu_k)^T \Sigma_k^{-1}(x_n - \mu_k) -\frac{1}{2} log|\Sigma_k| -\frac{D}{2} log(2\pi)$$

This part makes sense, however upon expansion it says we have this:

$$log(N(x_n|\mu_k, \Sigma_k)) = \mu_k^{T} \Sigma_k^{-1}x_n - \frac{1}{2} \mu_k^T \Sigma_k^{-1}\mu_k$$

I'm not sure how expanding the top function results in the bottom one, I'm fairly certain $-\frac{D}{2} log(2\pi)$ was removed because eventually we want to maximize over k but i'm not sure about the rest of it...

any help would be appreciated.

Edit: The formula for a multivariate gaussian is:

$$N(x_n|\mu_k, \Sigma_k) = \frac{1}{(2\pi)^{\frac{D}{2}} |\Sigma|^{\frac{1}{2}}} e^{- \frac{1}{2} (x_n - \mu_k)^T \Sigma^{-1}(x_n - \mu_k)}$$


Ignoring all the subscripts, and writing $A:=\Sigma^{-1}$, expand as follows: $$(x-\mu)^TA(x-\mu)=x^TAx-x^TA\mu-\mu^TAx+\mu^TA\mu$$ All the terms on the RHS are scalars since they have dimension $(1\times D)\times (D\times D)\times (D\times 1)$. Take the transpose of the second term and we see it's equal to the third term (since the covariance matrix $A$ is symmetric). This now simplifies to your form if we're throwing away (i.e., treating as constant) any term that doesn't involve $\mu$. (This would be appropriate if you were trying to maximize the log likelihood over $\mu$.)

  • $\begingroup$ That makes sense, but why is $-x^T A \mu$ missing from the formula then? because $\mu$ is included in it but it's not in the formula that i have $\endgroup$ – Ghazal Feb 28 '16 at 6:55
  • $\begingroup$ As mentioned above, $-x^TA\mu$ is the second term in the RHS. It equals its transpose, which is $-\mu^TAx$. Now add it to the third term in the RHS, to get $-2\mu^TAx$. Finally multiply through by $-1/2$ (which I've left out all along) to get $\mu^TAx-\frac12\mu^TA\mu$. $\endgroup$ – grand_chat Feb 28 '16 at 8:24
  • $\begingroup$ I see, thank you so much $\endgroup$ – Ghazal Feb 28 '16 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.