# Likelihood function of two i.i.d. normal samples

Let $$X_1,...,X_{n_1}$$ be an i.i.d. sample from $$N_p(\mu_1,\Sigma)$$ and let $$Y_1,...,Y_{n_2}$$ be an independent sample from $$N_p(\mu_2,\Sigma)$$, for some $$\mu_1,\mu_2 \in \mathbb{R}^p$$ and some invertible, $$p\times p$$ positive definite matrix $$\Sigma$$.

I'd like to find the likelihood function $$L(\mu_1,\mu_2,\Sigma)$$ of the commbined sample:

In my book, the likelihood function of $$X_1,...,X_n \sim N_p(\mu,\Sigma)$$ is given by

$$\frac{1}{(2\pi)^{np/2}\text{det}(\Sigma)^{n/2}}\exp\biggl(-1/2\bigl(\sum^n_{i=1}(x_i-\mu)^T\Sigma^{-1}(x_i-\mu)\bigr)\biggr)$$

So, $$L(\mu_1,\mu_2,\Sigma)=$$ $$\frac{1}{(2\pi)^{p(n_1+n_2)/2}\text{det}(\Sigma)^{\frac{n_1+n_2}{2}}}\exp\biggl(-1/2\sum^{n_1}_{i=1}(x_i-\mu_1)^T\Sigma^{-1}(x_i-\mu_1)-1/2\sum^{n_2}_{i=1}(y_i-\mu_2)^T\Sigma^{-1}(y_i-\mu_2)\biggr)$$

Would this be correct?

• @d.k.o. but otherwise everything else is okay? – user650626 Jan 15 at 10:34
• Almost. You need to multiply the second summation by $-1/2$ as well. – d.k.o. Jan 15 at 10:54

There is a simple trick that helps to deal with this likelihood. Set $$\bar{x}:=\frac{1}{n_1}\sum_{i=1}^{n_1}x_{i} \quad\text{and}\quad S_x:=\sum_{i=1}^{n_1}(x_{i}-\bar{x})(x_{i}-\bar{x})^{\top},$$ and, similarly, $$\bar{y}$$ and $$S_y$$. Then \begin{align} \ln\mathcal{L}(\mu_1,\mu_2,\Sigma)&=-\frac{(n_1+n_2)p}{2}\ln(2\pi)+\frac{(n_1+n_2)}{2}\ln|\Sigma^{-1}| \\ &\quad-\frac{1}{2}\operatorname{tr}(\Sigma^{-1}(S_x+S_y)) \\ &\quad-\frac{n_1}{2}(\bar{x}-\mu_1)^{\top}\Sigma^{-1}(\bar{x}-\mu_1)-\frac{n_2}{2}(\bar{y}-\mu_2)^{\top}\Sigma^{-1}(\bar{y}-\mu_2) \end{align} because \begin{align} &\sum_{i=1}^{n_1}(x_{i}-\mu)^{\top}\Sigma^{-1}(x_{i}-\mu_1)=\sum_{i=1}^{n_1}\operatorname{tr}\left((x_{i}-\mu_1)^{\top}\Sigma^{-1}(x_{i}-\mu_1)\right) \\ &\qquad=\sum_{i=1}^{n_1}\operatorname{tr}\left(\Sigma^{-1}(x_{i}-\mu_1)(x_{i}-\mu_1)^{\top}\right)=\operatorname{tr}\left(\Sigma^{-1}\sum_{i=1}^{n_1}(x_{i}-\mu_1)(x_{i}-\mu_1)^{\top}\right)\\ &\qquad=\operatorname{tr}\left(\Sigma^{-1}(S_x+n_1(\bar{x}-\mu_1)(\bar{x}-\mu_1)^{\top})\right) \\ &\qquad=\operatorname{tr}(\Sigma^{-1}S_x)+n_1(\bar{x}-\mu_1)^{\top}\Sigma^{-1}(\bar{x}-\mu_1). \end{align} Now it is easy to see that the likelihood is maximized when $$\mu_1=\bar{x}$$ and $$\mu_2=\bar{y}$$ ($$\because \Sigma^{-1}$$ is positive definite).
• shouldn't it be $-\frac{(n_1+n_2)}{2}\ln|\Sigma^{-1}|$ ? (emphasis on the "-"). Also, aren't we taking the log of the $\text{det}(\Sigma)$? not $\Sigma^{-1}$ – user650626 Jan 15 at 10:59
• @Maria No because I used the inverse of $\Sigma$ (it is useful because we can take the derivative w.r.t. $\Sigma^{-1}$). Btw, $|\Sigma^{-1}|$ is the determinant of $\Sigma^{-1}$. – d.k.o. Jan 15 at 11:33