Is Maximum likelihood estimation "better" than the method of moments?

Question

We are given a random sample $\mathbf x=(x_1,\dots,x_N)$ from a parameterized distribution $x_n\sim F_\theta$ and asked to estimate the parameters $\theta\in\Bbb R^m$.

Method of moments (MoM) estimates the parameters by equating the first $m$ sample moments to the corresponding theoretical moments of $F_\theta$ and then solving for the unknown parameters.

On the other hand, maximum likelihood estimation (MLE) estimates the parameters that maximize the (log-) likelihood function which usually correspond to the gradient of the log-likelihood equaling the zero vector.

I have read that maximum likelihood estimation is more efficient (asymptotically) than the method of moments but I don't know the reason why. Can someone please elaborate on this?

Answer 1

Here is an example where ML estimator is better than method of moments estimator.

Lets take uniform random variable: $U$ $\sim$ $[0,\theta]$.

Let $X_1,X_2,...,X_n$ be iid uniformly distributed samples following above uniform distribution.

Method of moments estimator : $$\theta_{est} = \frac{2}{n}\sum_i X_i$$

Estimator is unbiased i.e., $$E(\theta_{est}) = \theta$$

Variance: $$Var(\theta_{est}) = \frac{4}{n^2} n Var(X_1) = \frac{4}{n} \frac{\theta^2}{12} = \frac{\theta^2}{3n}$$

ML estimator : $$\theta_{est} = \frac{n+1}{n} \max_i X_i$$

Estimator is unbiased i.e., $$E(\theta_{est}) = \theta$$

Variance: $$Var(\theta_{est}) = \frac{(n+1)^2}{n^2}Var(\max_i X_i) = \frac{\theta^2}{n(n+2)}$$

In this case ML estimator is better then method of moments estimator as variance is clearly smaller. Also ML estimator is optimal asymptotically as it achieves Cramer Rao lower bound Asymptotically.

Intuition:

The reason why ML estimator performs better in above case is because ML estimator is based on sufficient statistics of the uniform distribution which is $T(X) = \max_i X_i$ whereas method of moments estimator is not based on sufficient statistics of the uniform distribution as it is based on sample mean $\sum_i X_i$. Estimates based on sufficient statistics are usually better due to Roa-Blackwell Theorem. But if the samples moments turns out to be sufficient statistics by chance then both ML and MoM estimator should perform reasonably. An example is normal distribution where both the ML and MoM estimates are based on sample mean which is the sufficient statistics and hence performs equally well.

For example By Roa-Blackwell Theorem, the estimator $$E(MoM \ estimate | T(X) (sufficient \ statistics))$$ is better than the MoM estimate. So atleast for sure, MoM estimate can be improved whenever its not based on sufficient statistics whereas ML is always based on sufficient statistics and not clear whether it can always be improved. Further as mentioned, ML estimator is asymptotically optimal as asymptotically it achieves Cramer Rao lower bound. So atleast we can say for sure as $n \rightarrow \infty$, ML estimate is better than MoM estimator, as whenever MoM estimator is not based on sufficient statistics, it can be improved by above explanation.