Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

The question is as follows, apologies in advance, I don't know how to do the LaTex thing in posts.

Let $X_1,\ldots,X_n, Y_1,\ldots,Y_n$ be independent random variables such that $X_i \sim N(\mu_1,\sigma^2)$ and $Y_j \sim N(\mu_2,\sigma^2)$. Both $\mu_1$ and $\mu_2$ are known but $\sigma^2$ is not.

Find the maximum likelihood estimator for $\sigma^2$ based on all $n+m$ observations. Show all working.

I am trying to work through with this but I am getting some horrible results when I get to the log-likelihood function. Any help in deriving the log-likelihood function would be appreciated.

Cheers.

share|improve this question
1  
Sounds a lot like a homework problem, and if so, pelase add the homework tag. Hint: Given observations $x_1,x_2,\ldots, x_m, y_1,y_2,\ldots,y_n$, replace each $x_i$ by $x_i-\mu_1$ and each $y_j$ by $y_j-\mu_2$. Now, you have $m+n$ observations of a zero-mean normal random variable of unknown variance $\sigma^2$. What is the likelihood function? What choice of value of $\sigma^2$ maximizes the likelihood function? Garnish and serve. –  Dilip Sarwate Mar 11 '13 at 2:59
    
Ok thanks for letting me know about the homework tag, I added that in. I can see what you are saying, but I am having trouble with writing down the likelihood function and taking the log of the likelihood, the product throws me out. –  Stephen Mar 11 '13 at 3:11
    
You mean you don't know a formula for the joint density of $m+n$ zero-mean independent normal random variables with common variance $\sigma^2$? Do you know the density of a single zero-mean normal random variable of variance $\sigma^2$? If so, can you use independence to figure out the rest? Also, maybe it isn't necessary to take logarithms? What is so hard about differentiating $av^{-m-n}\exp(-b/v)$ with respect to $v$ to figure out where the derivative is zero? –  Dilip Sarwate Mar 11 '13 at 3:19
    
I have given this question many attempts and still cannot work out how to get anything meaningful from what I have written. I think I have some gaps in my knowledge and despite by best attempts to bridge those gaps I don't really seem to be progressing. Approaching my lecturer has not been helpful. Anyways thank you for your suggestions, if you have anything else to add then please feel free to do so. –  Stephen Mar 11 '13 at 12:45

1 Answer 1

Since it's been a year... The likelihood of a single normal random variable $X_1$ is $L_1(\sigma^2 \mid x_1, \mu_1) = \frac{1}{\sqrt{2\pi \sigma^2}}\, \exp\left[-\frac{(x_1 - \mu_1)^2}{2\sigma^2} \right]$, and similar for $Y_1$ but using $\mu_2$ rather than $\mu_1$.

Since the $X_i$ and $Y_j$ are independent, we simply multiply the individual likelihoods to obtain the joint likelihood of $X_1,X_2,\ldots,X_n,Y_1,\ldots,Y_m$: $\begin{align} L(\sigma^2 \mid \mu_1,\mu_2,\boldsymbol x, \boldsymbol y) &=\prod_{i=1}^n \frac{1}{\sqrt{2\pi \sigma^2}}\, \exp\left[-\frac{(x_i - \mu_1)^2}{2\sigma^2} \right] \prod_{j=1}^m \frac{1}{\sqrt{2\pi \sigma^2}}\, \text{exp}\left[-\frac{(y_j - \mu_2)^2}{2\sigma^2} \right]\\ &= (2\pi\sigma^2)^{-\frac{m+n}{2}}\text{exp}\left[ -\frac{1}{2\sigma^2}\biggl(\sum_{i=1}^n(x_i - \mu_1)^2 + \sum_{j=1}^m(y_j - \mu_2)^2\biggr)\right] \end{align}$

The question of finding the maximum likelihood of $\sigma^2$ is now a simple maximisation of this function as a function of $\sigma^2$ (note that all other quantities are fixed and known). Take logs, then differentiate, then set to zero, then solve for $\sigma^2$.

share|improve this answer
1  
I revised the formatting to make the computation fit the page width. I also corrected some errors (you were missing a $\sigma^2$) and rewrote the likelihood to clarify that it is a function of $\sigma^2$ given $\mu_1, \mu_2, \boldsymbol x, \boldsymbol y$. –  heropup Apr 4 at 0:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.