1
$\begingroup$

I have the following Gaussian distribution:

$$p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma}) = \prod_{n=1}^N \mathcal{N}\left({\bf x}_n \mid \begin{bmatrix} \mu_1 \\ \mu_2 \\ a_1\mu_1+a_2\mu_2+a_0\end{bmatrix} , \begin{bmatrix}\sigma_1^2 & 0 & a_1\sigma_1^2\\ 0 & \sigma_2^2 & a_2\sigma_2^2 \\ a_1\sigma_1^2 & a_2\sigma_2^2 & a_1^2\sigma_1^2+a_2^2\sigma_2^2+\sigma_3^2\end{bmatrix}\right)$$

Now the idea is to compute the maximum likelihood w.r.t. $\mu_1$ and $a_1$. The log-likelihood is:

$$\ln p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma}) = -\frac{N}{2}\ln|\boldsymbol{\Sigma}|-\frac{1}{2}\sum_{n=1}^N ({\bf x}_n - \boldsymbol{\mu})^T\boldsymbol{\Sigma}^{-1}({\bf x}_n - \boldsymbol{\mu}) +\text{const.}\ $$

Taking the derivative w.r.t. $\boldsymbol{\mu}$ is not that hard,

$$\frac{\partial \ln p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma})}{\partial\boldsymbol{\mu}} = \sum_{n=1}^N({\bf x}_n-\boldsymbol{\mu})^T\boldsymbol{\Sigma}^{-1}\ .$$

Now I am not certain how to proceed, for example can I do:

$$\frac{\partial \ln p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma})}{\partial\mu_1}=\frac{\partial \ln p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma})}{\partial\boldsymbol{\mu}}\frac{\partial\boldsymbol{\mu}}{\partial\mu_1} = \begin{bmatrix}1 & 0 & a_1\end{bmatrix}\sum_{n=1}^N\boldsymbol{\Sigma}^{-1}({\bf x}_n-\boldsymbol{\mu}) = 0\ .$$

Which gives,

$$\begin{bmatrix}1\\ 0 \\ a_1\end{bmatrix}^T\boldsymbol{\Sigma}^{-1}\sum_{n=1}^N{\bf x}_n = \begin{bmatrix}1\\ 0 \\ a_1\end{bmatrix}^T\boldsymbol{\Sigma}^{-1}N\boldsymbol{\mu}\\ $$

I understand it is possible to solve for $\mu_1$, but seems very tedious and even more so when solving w.r.t. $a_1$ (Where we also need to account for the determinant in the log-likelihood).

Am I missing something here? i.e. is there an easier way to approach this?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.