0
$\begingroup$

This equation in image occurs while evaluating multivariate Gaussian distributions for log likelihood. I am short of proper linear algebra tools of how the first equation is reduced to the below one in the image.

Here we are using 2 different Gaussians with different mean, but same covariance. Can someone help me derive this with reference to matrix identities that are used?

Update:

The equation is:

$$\frac{1}{2}\left(\mathbf{x}-\boldsymbol{\mu}_{0}\right)^{T} \Sigma^{-1}\left(\mathbf{x}-\boldsymbol{\mu}_{0}\right)-\frac{1}{2}\left(\mathbf{x}-\boldsymbol{\mu}_{1}\right)^{T} \Sigma^{-1}\left(\mathbf{x}-\boldsymbol{\mu}_{1}\right)$$

It can be reduced to $\langle\mathbf{w}, \mathbf{x}\rangle+b$.

Where, $\mathbf{w}=\left(\boldsymbol{\mu}_{1}-\boldsymbol{\mu}_{0}\right)^{T} \Sigma^{-1}$ and $b=\frac{1}{2}\left(\boldsymbol{\mu}_{0}^{T} \Sigma^{-1} \boldsymbol{\mu}_{0}-\boldsymbol{\mu}_{1}^{T} \Sigma^{-1} \boldsymbol{\mu}_{1}\right)$

$\endgroup$

1 Answer 1

1
$\begingroup$

Because the covariance matrix $\ \Sigma\ $ and its inverse $\ \Sigma^{-1}\ $ are symmetric, then $$ \left(\mathbf{x}-\boldsymbol{\mu}_i\right)^T \Sigma^{-1} \left(\mathbf{x}-\boldsymbol{\mu}_i\right)=\mathbf{x}^T \Sigma^{-1} \mathbf{x}-2\boldsymbol{\mu}_i^T\Sigma^{-1} \mathbf{x}+ \boldsymbol{\mu}_i^T\Sigma^{-1} \boldsymbol{\mu}_i\ . $$ Thus, \begin{align} \left(\mathbf{x}-\boldsymbol{\mu}_0\right)^T \Sigma^{-1} &\left(\mathbf{x}-\boldsymbol{\mu}_0\right) -\left(\mathbf{x}-\boldsymbol{\mu}_1\right)^T \Sigma^{-1} \left(\mathbf{x}-\boldsymbol{\mu}_1\right)\\ &=-2 \boldsymbol{\mu}_0^T\Sigma^{-1} \mathbf{x}+ \boldsymbol{\mu}_0^T\Sigma^{-1} \boldsymbol{\mu}_0+ 2 \boldsymbol{\mu}_1^T\Sigma^{-1} \mathbf{x}-\boldsymbol{\mu}_1^T\Sigma^{-1} \boldsymbol{\mu}_1\\ &=2\left(\boldsymbol{\mu}_1-\boldsymbol{\mu}_0\right) \Sigma^{-1} \mathbf{x}+ \boldsymbol{\mu}_0^T\Sigma^{-1} \boldsymbol{\mu}_0-\boldsymbol{\mu}_1^T\Sigma^{-1} \boldsymbol{\mu}_1\\ &=2\left(\langle\mathbf{w},\mathbf{x}\rangle+b\right)\ . \end{align}

$\endgroup$
1
  • $\begingroup$ Thank you so much $\endgroup$
    – philophil
    Sep 28, 2020 at 13:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .