I have the following Gaussian distribution:

$$p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma}) = \prod_{n=1}^N \mathcal{N}\left({\bf x}_n \mid \begin{bmatrix} \mu_1 \\ \mu_2 \\ a_1\mu_1+a_2\mu_2+a_0\end{bmatrix} , \begin{bmatrix}\sigma_1^2 & 0 & a_1\sigma_1^2\\ 0 & \sigma_2^2 & a_2\sigma_2^2 \\ a_1\sigma_1^2 & a_2\sigma_2^2 & a_1^2\sigma_1^2+a_2^2\sigma_2^2+\sigma_3^2\end{bmatrix}\right)$$

Now the idea is to compute the maximum likelihood w.r.t. $\mu_1$ and $a_1$. The log-likelihood is:

$$\ln p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma}) = -\frac{N}{2}\ln|\boldsymbol{\Sigma}|-\frac{1}{2}\sum_{n=1}^N ({\bf x}_n - \boldsymbol{\mu})^T\boldsymbol{\Sigma}^{-1}({\bf x}_n - \boldsymbol{\mu}) +\text{const.}\ $$

Taking the derivative w.r.t. $\boldsymbol{\mu}$ is not that hard,

$$\frac{\partial \ln p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma})}{\partial\boldsymbol{\mu}} = \sum_{n=1}^N({\bf x}_n-\boldsymbol{\mu})^T\boldsymbol{\Sigma}^{-1}\ .$$

Now I am not certain how to proceed, for example can I do:

$$\frac{\partial \ln p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma})}{\partial\mu_1}=\frac{\partial \ln p({\bf X}\mid\boldsymbol{\mu}, \boldsymbol{\Sigma})}{\partial\boldsymbol{\mu}}\frac{\partial\boldsymbol{\mu}}{\partial\mu_1} = \begin{bmatrix}1 & 0 & a_1\end{bmatrix}\sum_{n=1}^N\boldsymbol{\Sigma}^{-1}({\bf x}_n-\boldsymbol{\mu}) = 0\ .$$

Which gives,

$$\begin{bmatrix}1\\ 0 \\ a_1\end{bmatrix}^T\boldsymbol{\Sigma}^{-1}\sum_{n=1}^N{\bf x}_n = \begin{bmatrix}1\\ 0 \\ a_1\end{bmatrix}^T\boldsymbol{\Sigma}^{-1}N\boldsymbol{\mu}\\ $$

I understand it is possible to solve for $\mu_1$, but seems very tedious and even more so when solving w.r.t. $a_1$ (Where we also need to account for the determinant in the log-likelihood).

Am I missing something here? i.e. is there an easier way to approach this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.