I am wondering how I can solve this problem more elegantly:

Let's say I have two set of i.i.d normal random variables with known variance $\sigma^2=1$: $$ X_1, X_2, ..., X_m \sim N(\mu_1, 1) $$ $$ Y_1, Y_2, ..., Y_n \sim N(\mu_2, 1) $$

And it's given that $\mu_1 \leq \mu_2$. So the MLE of such an observation should be:

$$ L = \prod_{i=1}^m f(X_i | \mu_1) \prod_{j=1}^n f(Y_j|\mu_2) $$ Where $f(\cdot|\mu)$ is the pdf of normal random variable with $\mu$ as mean and 1 as variance.

Under the constraint of $\mu_1\leq\mu_2$, I can think of this as an optimization problem, and that is to optimize for the L ( or log(L)), subject to the constraint. But this is pretty hard to reach at some kind of closed form solutions.

I am wondering if there is some more clever way to process this problem?


  • $\begingroup$ With or without the constraint you have the same likelihood surface which in this case has a single maximum. With a constraint you just look at a restricted area. So if $\bar{x}_1\leq \bar{x}_2$, then $\bar{x}_1$ and $\bar{x}_2$ are the mle's. Otherwise, the mle's are found on the border and the mle's for both $\mu_1$ and $\mu_2$ will be equal to $(m \bar{x}_1+n \bar{x}_2)/(m+n)$. $\endgroup$
    – JimB
    May 15, 2020 at 4:11

1 Answer 1


The joint likelihood of the parameters is actually $$\mathcal L(\mu_1, \mu_2 \mid \boldsymbol x, \boldsymbol y) \propto \exp\left(-\frac{m}{2}\left((\bar x - \mu_1)^2 + \hat \sigma_x^2\right) - \frac{n}{2}\left((\bar y - \mu_2)^2 + \hat \sigma_y^2 \right) \right) \mathbb 1 (\mu_1 \le \mu_2)$$ where $\bar x$, $\bar y$ are the respective sample means, and $\hat \sigma_x^2$, $\hat \sigma_y^2$ are the (biased) sample variances. After removing the remaining multiplicative factors and taking the logarithm, we get the log-likelihood on the sufficient statistics $$\ell (\mu_1, \mu_2 \mid \bar x, \bar y) \propto -\frac{m(\bar x - \mu_1)^2 + n(\bar y - \mu_2)^2}{2} + \log \mathbb 1 (\mu_1 \le \mu_2).$$ Since the first term is always negative unless $(\mu_1, \mu_2) = (\bar x, \bar y)$, this choice, when possible (i.e. $\bar x \le \bar y$), yields the maximum likelihood. If $\bar x > \bar y$, then this choice is not possible because $\log 0 = -\infty$. By considering the transformation $$(\mu_1', \mu_2') = (\sqrt{m} \mu_1, \sqrt{n} \mu_2),$$ we seek to minimize the squared distance $$(\mu_1' - \sqrt{m}\bar x)^2 + (\mu_2' - \sqrt{n} \bar y)^2$$ subject to the constraint $\mu_1'/\sqrt{m} \le \mu_2'/\sqrt{n}$. As the level sets of this function of $(\mu_1', \mu_2')$ comprise circles centered at $(\sqrt{m} \bar x, \sqrt{n} \bar y)$, we observe that the smallest radius occurs for a circle tangent to the boundary of the transformed parameter space $$\mu_1'/\sqrt{m} = \mu_2'/\sqrt{n}.$$ Back-transforming shows that this corresponds to the boundary of the original parameter space, $\mu_1 = \mu_2$. Now we can solve for the constrained MLE via simple differentiation. The critical points satisfy $$0 = \frac{d \ell(\mu_1, \mu_1 \mid \bar x, \bar y)}{d \mu_1} = \mu_1 (m+n) - (m \bar x + n \bar y),$$ hence the MLE is $$(\hat \mu_1, \hat \mu_2) = \begin{cases} (\bar x, \bar y), & \bar x \le \bar y \\ \left(\frac{m \bar x + n \bar y}{m+n}, \frac{m \bar x + n \bar y}{m+n}\right), & \bar x > \bar y. \end{cases}$$


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .