0
$\begingroup$

If we have two normal distributions: $X_1,\dots,X_n; X\sim N(\mu_1,\sigma^2)$ and $Y_1,\dots,Y_m; Y\sim N(\mu_2,\sigma^2)$, what is the maximum likelihood estimator of $\sigma^2$ using both samples.

Both are normal distributions. I only calculate $X\sim N$ and will apply the results to $Y\sim N$.

$$X\sim N(\mu_1,\sigma^2)\implies f_X=\displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(x-\mu_1)/2\sigma^2}.$$

The likelihood is given by

\begin{align}L(\mu_1,\sigma^2)&=\displaystyle\prod_{i=1}^n \displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(x_i-\mu_1)/2\sigma^2}\\&=\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma} \right)^n \displaystyle\prod_{i=1}^n e^{-(x_i-\mu_1)/2\sigma^2}\\&=\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^n e^{\sum_{i=1}^n -(x_i-\mu_1)/2\sigma^2}. \end{align}

Taking logarithms

\begin{align}\ln(L(\mu_1,\sigma^2)&=\ln\left(\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^n e^{\sum_{i=1}^n -(x_i-\mu_1)/2\sigma^2} \right)\\& = \ln\left( \left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^n \right) + \ln\left( e^{\sum_{i=1}^n -(x_i-\mu_1)/2\sigma^2} \right)\\&=n\ln\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right) + \displaystyle\sum_{i=1}^n -(x_i-\mu_1)/2\sigma^2 \\&=-n\ln(\sqrt{2\pi})- n\ln(\sigma) - \displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)}{2\sigma^2}. \end{align}

Now to get MLE we have $\partial_{\sigma}L = -\displaystyle\frac{n}{\sigma}+\displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)}{2\sigma^3}$ which is $0$ only if $\displaystyle\frac{n}{\sigma}=\displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)}{2\sigma^3}\implies\sigma^2=\frac{1}{n}\displaystyle\sum_{i=1}^n (x_i-\mu_1)=\frac{1}{n}\displaystyle\sum_{i=1}^n (x_i-\bar x)$.

Then for $Y\sim N$ is also $\displaystyle\sigma^2=\frac{1}{m}\displaystyle\sum_{j=1}^m (y_j-\bar y)$.

But the problem says I need to use both samples, which sounds like I need $\sigma^2$ that works for both. Can I combine the results in any way?

$\endgroup$
2
  • $\begingroup$ Are the samples independent? Then you can multiply the likelihoods of the $X$ sample and the $Y$ sample to get a combined likelihood $L(\mu_1, \mu_2, \sigma^2)$ $\endgroup$ – FullofDill Sep 14 '17 at 3:26
  • $\begingroup$ @FullofDill Yes they are. Thank you. $\endgroup$ – Cure Sep 14 '17 at 3:59
1
$\begingroup$

First, you miss out a square.

$$X\sim N(\mu_1,\sigma^2)\implies f_X=\displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(x-\mu_1)^{\color{red}{2}}/2\sigma^2}.$$

Assuming independence, the likelihood is given by

\begin{align}L(\mu_1, \mu_2, \sigma^2)&=\left(\displaystyle\prod_{i=1}^n \displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(x_i-\mu_1)^2/2\sigma^2} \right)\left(\displaystyle\prod_{j=1}^m \displaystyle\frac{1}{\sqrt{2\pi}\sigma}e^{-(y_j-\mu_2^2)/2\sigma^2} \right)\\&=\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma} \right)^{m+n} \left( \displaystyle\prod_{i=1}^n e^{-(x_i-\mu_1)^2/2\sigma^2} \right)\left( \displaystyle\prod_{j=1}^m e^{-(y_j-\mu_2)^2/2\sigma^2} \right)\\&=\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^{m+n} e^{\sum_{i=1}^n \frac{-(x_i-\mu_1)^2}{2\sigma^2} + \sum_{j=1}^m \frac{-(y_j-\mu_2)^2}{2\sigma^2} }\end{align}

Taking logarithms

\begin{align}\ln(L(\mu_1,\mu_2, \sigma^2)&=\ln\left(\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^{m+n} e^{\sum_{i=1}^n \frac{-(x_i-\mu_1)^2}{2\sigma^2} + \sum_{j=1}^m \frac{-(y_j-\mu_2)^2}{2\sigma^2} } \right)\\& = \ln\left( \left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right)^{m+n} \right) + \ln\left( e^{\sum_{i=1}^n \frac{-(x_i-\mu_1)^2}{2\sigma^2} +\sum_{j=1}^m \frac{-(y_j-\mu_2)^2}{2\sigma^2} } \right)\\&=(m+n)\ln\left(\displaystyle\frac{1}{\sqrt{2\pi}\sigma}\right) + \displaystyle\sum_{i=1}^n \frac{-(x_i-\mu_1)^2}{2\sigma^2} + \displaystyle\sum_{j=1}^m \frac{-(y_j-\mu_2)^2}{2\sigma^2} \\&=-(m+n)\ln(\sqrt{2\pi})- (m+n)\ln(\sigma) - \displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)^2}{2\sigma^2}- \displaystyle\sum_{j=1}^m \frac{(y_j-\mu_2)^2}{2\sigma^2}\end{align}

Now to get MLE we have $\partial_{\sigma}\ln L = -\displaystyle\frac{m+n}{\sigma}+\displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)^2}{\sigma^3}+\displaystyle\sum_{i=1}^n \frac{(y_j-\mu_2)^2}{\sigma^3}$ which is $0$ only if $\displaystyle\frac{m+n}{\sigma}=\displaystyle\sum_{i=1}^n \frac{(x_i-\mu_1)^2}{\sigma^3} + \displaystyle\sum_{j=1}^m \frac{(y_j-\mu_2)^2}{\sigma^3}\implies\sigma^2=\frac{1}{m+n} \left(\displaystyle\sum_{i=1}^n (x_i-\mu_1)^2 + \displaystyle\sum_{j=1}^m (y_j-\mu_2)^2\right)$.

$\endgroup$
3
  • $\begingroup$ Hi. It is a likelihood of three variables, why we don't need to check the Hessian Matrix? $\endgroup$ – Nan Nov 20 '17 at 1:24
  • $\begingroup$ Sure, we should check the Hessian matrix to be sure. $\endgroup$ – Siong Thye Goh Nov 20 '17 at 1:41
  • $\begingroup$ Thank you very much! But I checked the determinant is zero at this point with $\hat{u_{X}}=\bar{x}$ and $\hat{u_{Y}}=\bar{y}$. So why is that? $\endgroup$ – Nan Nov 20 '17 at 1:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.