Consider the following problem:

In a biology experiment, we list the observed frequencies of 4 genetic classes and assume the genetic probability of each class. The information is given in the table below:

\begin{array}{|c|c|c|c|} \hline Genetic \; class & Observed \; frequency & Probability \\ \hline AB & n_1 & \frac{2+\theta}{4} \\ \hline Ab & n_2 & \frac{1-\theta}{4} \\ \hline aB & n_3 & \frac{1-\theta}{4} \\ \hline ab & n_4 & \frac{\theta}{4} \\ \hline \end{array}

I preliminary observation is that $\theta \in [0,1]$.

  1. Determine the MLE of $\theta$.

  2. For the case of $n_i \to \infty$ compute the variance of the MLE.

My problems

I have problems with the first part since there is not a single functional form. The equation depends on the value. Should I infer a proper distribution?

I'd say the second part follows for some asymptotic result but without the first I cannot proceed.


Apparently, this is a variation of problem 3.18 in Examples in Parametric Inference with R by Ulhas Jayram Dixit.

My progress

  1. Assuming a multinomial distribution I get that $f(\theta|x) \propto \Big(\frac{2+\theta}{4}\Big)^{n_1}\Big(\frac{1-\theta}{4}\Big)^{n_2+n_3}\Big(\frac{\theta}{4}\Big)^{n_4}$ with $\theta \in ]0,1[$.

Then the likelihood equation gives $\frac{d}{d\theta} log f(\theta|x) = \frac{n_1}{2+\theta}-\frac{n_2+n_3}{1-\theta}+\frac{n_4}{\theta}$. Setting this to zero gives (as valid value in ]0,1[) $\theta = -1 + \sqrt{(\frac{n_1-2n_2-2n_3-n_4}{4n_4})^2 + \frac{n}{2n_4}}$. The second derivative gives an always negative value so this must be a relative maximum. Given that in the extremes the likelihood is zero, the value has to be a global maximum.

  1. Asymptotic normality of mle tells me that $\hat \theta_n \approx N(\theta_0,\frac{1}{I_{\overline{x}}(\theta_0)})$ where $\theta_n$ is the mle sequence and $\theta_0$ is the real value of the parameter. I stress that its variance is the information of the whole sample vector. Since $I_{\overline{x}}(\theta_0) = Var_{X|\theta_0} S(X|\theta_0)$, I wonder if it is legal to approximate $Var_{X|\theta_0} S(X|\theta_0) \approx Var_{X|\hat \theta_n} S(X|\hat \theta_n)$ as the value of $\theta_0$ is unknown...
  • 2
    $\begingroup$ The MLE maximizes what function of $\theta$? Write a formula for that function, and try to see if the calculus recipe for maximizing it works. $\endgroup$ – kimchi lover Jan 6 '18 at 20:28

If I told you that $X$ follows a categorical distribution with $m = 4$ categories with probabilities $(p_1, p_2, p_3, p_4)$--that is to say, $$\Pr[X = i] = \begin{cases} p_i, & i \in \{1, 2, 3, 4\}, \\ 0, & \text{otherwise}, \end{cases}$$ then how would you find the MLE of, say, $p_1$ for a sample $\boldsymbol X$?

The same idea applies to your case, except instead of estimating a single parameter out of (essentially) three parameters (the fourth is determined from the others), all four categorical probabilities are related through the single parameter $\theta$. Consequently, the first thing to do is to write the likelihood function for $\theta$ given the sample.

Note that for any categorical distribution, a sufficient statistic for the joint probability parameter vector $\boldsymbol p = (p_1, \ldots, p_m)$ is the frequency of the respective categories observed in your sample. For example, if there are four categories, then the sample $$\boldsymbol x = (1, 2, 1, 1, 3, 1, 2, 2, 3, 1, 4)$$ can be reduced to $$\boldsymbol T(\boldsymbol x) = (5, 3, 2, 1),$$ where the $i^{\rm th}$ position counts the number of occurrences of the $i^{\rm th}$ categorical outcome. You may be able to achieve even further data reduction when the categorical probabilities are related through $\theta$, but $\boldsymbol T$ is a start for you to figure out how to write $$\mathcal L(\theta \mid \boldsymbol x) = \mathcal L (\theta \mid \boldsymbol T(\boldsymbol x)).$$

  • $\begingroup$ thanks @heropup do you think that the approximation i do for the second part is good? (see my progress->2 in the question) $\endgroup$ – Rodrigo Jan 7 '18 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.