# Maximum likelihood for two samples

If we have two normal distributions: $X_1,\dots,X_n; X\sim N(\mu_1,\sigma^2)$ and $Y_1,\dots,Y_m; Y\sim N(\mu_2,\sigma^2)$, what is the maximum likelihood estimator of $\sigma^2$ using both samples.

Both are normal distributions. I only calculate $X\sim N$ and will apply the results to $Y\sim N$.

$$X\sim N(\mu_1,\sigma^2)\implies f_X=\displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(x-\mu_1)/2\sigma^2}.$$

The likelihood is given by

\begin{align}L(\mu_1,\sigma^2)&=\displaystyle\prod_{i=1}^n \displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(x_i-\mu_1)/2\sigma^2}\\&=\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma} \right)^n \displaystyle\prod_{i=1}^n e^{-(x_i-\mu_1)/2\sigma^2}\\&=\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^n e^{\sum_{i=1}^n -(x_i-\mu_1)/2\sigma^2}. \end{align}

Taking logarithms

\begin{align}\ln(L(\mu_1,\sigma^2)&=\ln\left(\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^n e^{\sum_{i=1}^n -(x_i-\mu_1)/2\sigma^2} \right)\\& = \ln\left( \left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^n \right) + \ln\left( e^{\sum_{i=1}^n -(x_i-\mu_1)/2\sigma^2} \right)\\&=n\ln\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right) + \displaystyle\sum_{i=1}^n -(x_i-\mu_1)/2\sigma^2 \\&=-n\ln(\sqrt{2\pi})- n\ln(\sigma) - \displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)}{2\sigma^2}. \end{align}

Now to get MLE we have $\partial_{\sigma}L = -\displaystyle\frac{n}{\sigma}+\displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)}{2\sigma^3}$ which is $0$ only if $\displaystyle\frac{n}{\sigma}=\displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)}{2\sigma^3}\implies\sigma^2=\frac{1}{n}\displaystyle\sum_{i=1}^n (x_i-\mu_1)=\frac{1}{n}\displaystyle\sum_{i=1}^n (x_i-\bar x)$.

Then for $Y\sim N$ is also $\displaystyle\sigma^2=\frac{1}{m}\displaystyle\sum_{j=1}^m (y_j-\bar y)$.

But the problem says I need to use both samples, which sounds like I need $\sigma^2$ that works for both. Can I combine the results in any way?

• Are the samples independent? Then you can multiply the likelihoods of the $X$ sample and the $Y$ sample to get a combined likelihood $L(\mu_1, \mu_2, \sigma^2)$ – FullofDill Sep 14 '17 at 3:26
• @FullofDill Yes they are. Thank you. – Cure Sep 14 '17 at 3:59

## 1 Answer

First, you miss out a square.

$$X\sim N(\mu_1,\sigma^2)\implies f_X=\displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(x-\mu_1)^{\color{red}{2}}/2\sigma^2}.$$

Assuming independence, the likelihood is given by

\begin{align}L(\mu_1, \mu_2, \sigma^2)&=\left(\displaystyle\prod_{i=1}^n \displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(x_i-\mu_1)^2/2\sigma^2} \right)\left(\displaystyle\prod_{j=1}^m \displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(y_j-\mu_2^2)/2\sigma^2} \right)\\&=\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma} \right)^{m+n} \left( \displaystyle\prod_{i=1}^n e^{-(x_i-\mu_1)^2/2\sigma^2} \right)\left( \displaystyle\prod_{j=1}^m e^{-(y_j-\mu_2)^2/2\sigma^2} \right)\\&=\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^{m+n} e^{\sum_{i=1}^n \frac{-(x_i-\mu_1)^2}{2\sigma^2} + \sum_{j=1}^m \frac{-(y_j-\mu_2)^2}{2\sigma^2} }\end{align}

Taking logarithms

\begin{align}\ln(L(\mu_1,\mu_2, \sigma^2)&=\ln\left(\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^{m+n} e^{\sum_{i=1}^n \frac{-(x_i-\mu_1)^2}{2\sigma^2} + \sum_{j=1}^m \frac{-(y_j-\mu_2)^2}{2\sigma^2} } \right)\\& = \ln\left( \left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^{m+n} \right) + \ln\left( e^{\sum_{i=1}^n \frac{-(x_i-\mu_1)^2}{2\sigma^2} +\sum_{j=1}^m \frac{-(y_j-\mu_2)^2}{2\sigma^2} } \right)\\&=(m+n)\ln\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right) + \displaystyle\sum_{i=1}^n \frac{-(x_i-\mu_1)^2}{2\sigma^2} + \displaystyle\sum_{j=1}^m \frac{-(y_j-\mu_2)^2}{2\sigma^2} \\&=-(m+n)\ln(\sqrt{2\pi})- (m+n)\ln(\sigma) - \displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)^2}{2\sigma^2}- \displaystyle\sum_{j=1}^m \frac{(y_j-\mu_2)^2}{2\sigma^2}\end{align}

Now to get MLE we have $\partial_{\sigma}\ln L = -\displaystyle\frac{m+n}{\sigma}+\displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)^2}{\sigma^3}+\displaystyle\sum_{i=1}^n \frac{(y_j-\mu_2)^2}{\sigma^3}$ which is $0$ only if $\displaystyle\frac{m+n}{\sigma}=\displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)^2}{\sigma^3} + \displaystyle\sum_{j=1}^m \frac{(y_j-\mu_2)^2}{\sigma^3}\implies\sigma^2=\frac{1}{m+n} \left(\displaystyle\sum_{i=1}^n (x_i-\mu_1)^2 + \displaystyle\sum_{j=1}^m (y_j-\mu_2)^2\right)$.

• Hi. It is a likelihood of three variables, why we don't need to check the Hessian Matrix? – Nan Nov 20 '17 at 1:24
• Sure, we should check the Hessian matrix to be sure. – Siong Thye Goh Nov 20 '17 at 1:41
• Thank you very much! But I checked the determinant is zero at this point with $\hat{u_{X}}=\bar{x}$ and $\hat{u_{Y}}=\bar{y}$. So why is that? – Nan Nov 20 '17 at 1:51