1
$\begingroup$

Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some invertible, $p\times p$ positive definite matrix $\Sigma$.

I'd like to find the likelihood function $L(\mu_1,\mu_2,\Sigma)$ of the commbined sample:

In my book, the likelihood function of $X_1,...,X_n \sim N_p(\mu,\Sigma)$ is given by

$$\frac{1}{(2\pi)^{np/2}\text{det}(\Sigma)^{n/2}}\exp\biggl(-1/2\bigl(\sum^n_{i=1}(x_i-\mu)^T\Sigma^{-1}(x_i-\mu)\bigr)\biggr)$$

So, $$L(\mu_1,\mu_2,\Sigma)=$$ $$\frac{1}{(2\pi)^{p(n_1+n_2)/2}\text{det}(\Sigma)^{\frac{n_1+n_2}{2}}}\exp\biggl(-1/2\sum^{n_1}_{i=1}(x_i-\mu_1)^T\Sigma^{-1}(x_i-\mu_1)-1/2\sum^{n_2}_{i=1}(y_i-\mu_2)^T\Sigma^{-1}(y_i-\mu_2)\biggr)$$

Would this be correct?

$\endgroup$
2
  • $\begingroup$ @d.k.o. but otherwise everything else is okay? $\endgroup$ – user650626 Jan 15 at 10:34
  • $\begingroup$ Almost. You need to multiply the second summation by $-1/2$ as well. $\endgroup$ – d.k.o. Jan 15 at 10:54
0
$\begingroup$

There is a simple trick that helps to deal with this likelihood. Set $$ \bar{x}:=\frac{1}{n_1}\sum_{i=1}^{n_1}x_{i} \quad\text{and}\quad S_x:=\sum_{i=1}^{n_1}(x_{i}-\bar{x})(x_{i}-\bar{x})^{\top}, $$ and, similarly, $\bar{y}$ and $S_y$. Then \begin{align} \ln\mathcal{L}(\mu_1,\mu_2,\Sigma)&=-\frac{(n_1+n_2)p}{2}\ln(2\pi)+\frac{(n_1+n_2)}{2}\ln|\Sigma^{-1}| \\ &\quad-\frac{1}{2}\operatorname{tr}(\Sigma^{-1}(S_x+S_y)) \\ &\quad-\frac{n_1}{2}(\bar{x}-\mu_1)^{\top}\Sigma^{-1}(\bar{x}-\mu_1)-\frac{n_2}{2}(\bar{y}-\mu_2)^{\top}\Sigma^{-1}(\bar{y}-\mu_2) \end{align} because \begin{align} &\sum_{i=1}^{n_1}(x_{i}-\mu)^{\top}\Sigma^{-1}(x_{i}-\mu_1)=\sum_{i=1}^{n_1}\operatorname{tr}\left((x_{i}-\mu_1)^{\top}\Sigma^{-1}(x_{i}-\mu_1)\right) \\ &\qquad=\sum_{i=1}^{n_1}\operatorname{tr}\left(\Sigma^{-1}(x_{i}-\mu_1)(x_{i}-\mu_1)^{\top}\right)=\operatorname{tr}\left(\Sigma^{-1}\sum_{i=1}^{n_1}(x_{i}-\mu_1)(x_{i}-\mu_1)^{\top}\right)\\ &\qquad=\operatorname{tr}\left(\Sigma^{-1}(S_x+n_1(\bar{x}-\mu_1)(\bar{x}-\mu_1)^{\top})\right) \\ &\qquad=\operatorname{tr}(\Sigma^{-1}S_x)+n_1(\bar{x}-\mu_1)^{\top}\Sigma^{-1}(\bar{x}-\mu_1). \end{align} Now it is easy to see that the likelihood is maximized when $\mu_1=\bar{x}$ and $\mu_2=\bar{y}$ ($\because \Sigma^{-1}$ is positive definite).

$\endgroup$
2
  • $\begingroup$ shouldn't it be $-\frac{(n_1+n_2)}{2}\ln|\Sigma^{-1}|$ ? (emphasis on the "-"). Also, aren't we taking the log of the $\text{det}(\Sigma)$? not $\Sigma^{-1}$ $\endgroup$ – user650626 Jan 15 at 10:59
  • $\begingroup$ @Maria No because I used the inverse of $\Sigma$ (it is useful because we can take the derivative w.r.t. $\Sigma^{-1}$). Btw, $|\Sigma^{-1}|$ is the determinant of $\Sigma^{-1}$. $\endgroup$ – d.k.o. Jan 15 at 11:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy