Here is a question:

We have a machine A that functions with probability $\theta_1 \theta_2$, and a machine B that functions with probability $\theta_1 \theta_2^2$. A random sample of $n$ A machines and an independent random sample of $n$ B machines are selected. Of these, $n_1$ and $n_2$ function respectively. Find the MLEs of $\theta_1$ and $\theta_2$.

I have problems understand the question when it says $n_1$ and $n_2$. Does that mean each type of machine samples have $2$ out of $n$ working? and is there any difference we need to draw between Sample A and B since sample B is independent?

Thanks a lot!

  • $\begingroup$ Yes I can confirm that. Thanks! $\endgroup$ – Febday Jan 30 '17 at 15:11
  • $\begingroup$ Cool! You should take a look at MathJax - If you are planning on staying around, as this makes your posts more likely to be answered by the population on this site :). $\endgroup$ – Chinny84 Jan 30 '17 at 15:16

Your likelihood function looks like this


Maximizing log likelihood we can find that

$$\theta_1\theta_2^2= \frac{n_2} n$$

$$\theta_1\theta_2= \frac{n_1} n$$

From here I think we can infer that $\theta_2=n_2/n_1$ and $\theta_1=n_1^2/nn_2$

If someone could check my solution I would appreciate that.

  • $\begingroup$ This looks right but I'd have said more than this. $\endgroup$ – Michael Hardy Jan 30 '17 at 16:59
  • $\begingroup$ Do you mean it is unclear how to derive that $\theta_1\theta_2^2= \frac{n_2} n$ and $\theta_1\theta_2= \frac{n_1} n$? $\endgroup$ – Markoff Chainz Jan 30 '17 at 17:03
  • $\begingroup$ Why doesn't the likelihood function involve $n$ chooses $n_1$ and $n$ chooses $n_2$? $\endgroup$ – Febday Jan 30 '17 at 17:12
  • $\begingroup$ Because we dont need the probability of any $n_1$ machines functioning from $n$. We have exact sample of $n_1$ and $n_2$ units $\endgroup$ – Markoff Chainz Jan 30 '17 at 17:16
  • $\begingroup$ @Febday : Because likelihood differing only by a constant positive factor are equivalent. And "constant" in this case means not depending on $(\theta_1,\theta_2).\qquad$ $\endgroup$ – Michael Hardy Jan 30 '17 at 17:21

You have the likelihood function $$ L(\theta_1,\theta_2) = \text{constant} \times (\theta_1\theta_2)^{n_1} (1-\theta_1\theta_2)^{n-n_1} (\theta_1\theta_2^2)^{n_2}(1-\theta_1\theta_2^2)^{n-n_2} \tag 1 $$ where "constant" means not depending on $\theta_1$ or $\theta_2$.

It start by letting $\alpha=\theta_1 \theta_2$ and $\beta = \theta_1 \theta_2^2.$ That transforms $(1)$ to $$ \alpha^{n_1} (1-\alpha)^{n-n_1} \beta^{n_2} (1-\beta)^{n-n_2}. $$ The logarithm of this expression is $$ \ell = n_1 \log \alpha + (n-n_1)\log(1-\alpha) + n_2\log\beta + (n-n_2)\log(1- \beta). $$ So we have $$ \frac{\partial\ell}{\partial\alpha} = \frac{n_1} \alpha - \frac{n-n_1}{1-\alpha}. $$ This is $0$ precisely when $\alpha = \dfrac{n_1}n$. By symmetry we similarly see that $\partial\ell/\partial\beta=0$ when $\beta = \dfrac{n_2} n.$

Given \begin{align} \theta_1\theta_2 & = \alpha = \frac{n_1} n \tag 2 \\[10pt] \text{and } \theta_1\theta_2^2 & = \beta = \frac{n_2} n \tag 3 \end{align} we can divide the left side of $(3)$ by the left side of $(2)$ to get $\theta_2,$ and doing the same with the right sides we get $\theta_2=n_2/n_1.$ We can divide the square of the left side of $(2)$ by the square of the left side of $(3)$ to get $\theta_1$, and then doing the same with the right sides we have $\theta_1 = n_1^2/n_2 n.$

Here we have used what you often see called the "invariance" of maximum-likelihood estimates, but it's really equivariance rather than invariance.

Note also the mere fact of the derivative being $0$ does not prove that there is a global maximum. In this case the function of either $\alpha$ or $\beta$ is $0$ at the two extreme points $0$ and $1$, and is positive between those two extremes, and is continuous. That proves there is a global maximum somewhere strictly between $0$ and $1$. The function is also everywhere differentiable, so the derivative must be $0$ at a non-endpoint maximum. And then we find that there is only one point where the derivative is $0$ and we can conclude that's where the global maximum is.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.