Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Suppose that $X_1,\ldots,X_n$ are normal with mean $\mu_1$; $Y_1,\ldots,Y_n$ are normal with mean $\mu_2$; and $W_1,\ldots,W_n$ are normal with mean $\mu_1+\mu_2$. Assuming that all $3n$ random variables are independent, with a common variance, find the maximum likelihood estimators of $\mu_1$ and $\mu_2$.

Solving for $\mu_1$ using $X_1,\ldots,X_n$ I got $\mu_1$ = the sample mean of $X$. Similarly, solving for $\mu_2$ using $Y_1,\ldots,Y_2$ I got $\mu_2$ = the sample mean of $Y$. However, I'm not sure if that's what this question meant, especially since I don't understand what the purpose of giving the mean of $W_1,\ldots,W_n$ is.

share|improve this question
    
Maybe your title should say "Maximum likelihood estimators of the means of three independent normal random variables with a common variance". –  Michael Hardy Nov 19 '12 at 14:46

2 Answers 2

The likelihood function based on $X_1,\ldots,X_n,Y_1,\ldots,Y_n,W_1,\ldots,W_n$ is given by $$ L(\mu_1,\mu_2,\sigma^2)=\left[\prod_{i=1}^n \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{1}{2\sigma^2}(x_i-\mu_1)^2\right)\right]\times\left[\prod_{i=1}^n \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{1}{2\sigma^2}(y_i-\mu_2)^2\right)\right]\times \left[\prod_{i=1}^n \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{1}{2\sigma^2}(w_i-(\mu_1+\mu_2))^2\right)\right] $$

Then the log-likelihood function is $$ l(\mu_1,\mu_2,\sigma^2)=-\frac{3n}{2}\log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\left(\sum_{i=1}^n(x_i-\mu_1)^2+\sum_{i=1}^n(y_i-\mu_2)^2+\sum_{i=1}^n(w_i-\mu_1-\mu_2)^2\right) $$ and so $$ \frac{\partial l}{\partial \sigma^2}(\mu_1,\mu_2,\sigma^2)=-\frac{3n}{2\sigma^2}+\frac{1}{2(\sigma^2)^2}\left(\sum_{i=1}^n(x_i-\mu_1)^2+\sum_{i=1}^n(y_i-\mu_2)^2+\sum_{i=1}^n(w_i-\mu_1-\mu_2)^2\right). $$ This is equal to $0$ if and only if $$ \sigma^2=\hat{\sigma}^2=\frac{1}{3n}\left(\sum_{i=1}^n(x_i-\mu_1)^2+\sum_{i=1}^n(y_i-\mu_2)^2+\sum_{i=1}^n(w_i-\mu_1-\mu_2)^2\right). $$ Thus $$ l(\mu_1,\mu_2,\sigma^2)\leq l(\mu_1,\mu_2,\hat{\sigma}^2)=-\frac{3n}{2}\log(2\pi\hat{\sigma}^2)-\frac{3n}{2}. $$ Now maximize this expression with respect to $\mu_1$ and $\mu_2$.

share|improve this answer
    
I suspect it was not intended that $\sigma$ would be taken to be known. So you might want to include $\sigma$ as one of the arguments to $L$, and possibly try to show that the value of $\sigma$ that maximizes $L$ does not depend on $\mu_1$ and $\mu_2$. –  Michael Hardy Nov 19 '12 at 14:45
    
@MichaelHardy: Hmm, according to this, the maximum value of $\sigma$ does depend on $\mu_1$ and $\mu_2$, but is this a problem? –  Stefan Hansen Nov 19 '12 at 15:24
    
I was momentarily confused. The values of $\mu_1$ and $\mu_2$ that maximize the likelihood as a function of those two variables and $\sigma$ does not depend on $\sigma$. Hence one may substitute those two MLEs for the variables and then find the value of $\sigma$ that maximizes $L$. So we don't really need to include $\sigma$ among the variables. But if you were going to do likelihood ratio tests on hypotheses about $\mu_1$ and $\mu_2$, you'd probably still want to find the MLE for $\sigma$. –  Michael Hardy Nov 19 '12 at 16:55
    
Aha, I see your point :) –  Stefan Hansen Nov 19 '12 at 16:59

It is useful to remember that $$ \sum_{i=1}^n (x_i-\mu)^2 = n(\overline{x}-\mu)^2 + \sum_{i=1}^n (x_i-\overline{x})^2,\text{ where }\overline{x} = \frac{x_1+\cdots+x_n}{n}. $$ Then you can rely on the fact that $\mu$ appears only in the first term on the right. The value of $\mu$ that minimizes this is therefore the value that minimizes the first term. That is $\hat\mu=\overline{x}$.

In the present problem we need to minimize $$ \sum_{i=1}^n (x_i-\mu_1)^2 + \sum_{i=1}^n (y_i-\mu_2)^2 + \sum_{i=1}^n (w_i-(\mu_1+\mu_2))^2 $$ $$ = n(\overline{x}-\mu_1)^2 + n(\overline{y}-\mu_2)^2 + n(\overline{w}-(\mu_1+\mu_2))^2+\text{terms not depending on $\mu_1$ or $\mu_2$}. $$ $$ = n\left(2\mu_1^2+2\mu_2^2 + 2\mu_1\mu_2-2\mu_1(\overline{w}+\overline{x})-2\mu_2(\overline{w}+\overline{y}) + \text{terms not depending on $\mu_1$ or $\mu_2$}\right). $$ Next you could talk about a rotation in the $(\mu_1,\mu_2)$ plane, but maybe it's more efficient at this point just to find partial derivatives with respect to $\mu_1$ and $\mu_2$ and set them to $0$ and solve for those two variables.

I get $$ \mu_2=\overline{w}+\text{a certain weighted average of $\overline{x}$ and $\overline{y}$}. $$ A similar thing will apply to $\mu_1$.

share|improve this answer
    
Now I'm a bit suspicious because I'm wondering if it shouldn't have been a weighted average of all three..... –  Michael Hardy Nov 19 '12 at 17:16

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.