4
$\begingroup$

Method of Moments - What's the logic? We have a random vector $X=(X_1,X_2,...,X_n)$ that generates a sample. The hypothesis is that our components of the random vector $X_i$ are i.i.d.

The method of moments, for point estimation, in statistical inference, suggests a way to find an estimate of a parameter $\theta$. If you need to estimate only one parameter, the procedure seems to force you to equate the first theoretical moment $E(X_i)$ to the corresponding sample moment.

Question:

If I have only one parameter to estimate, can I equate the second theoretical moment to the second sample moment in order to find $\theta$? Does the fact that I have only one parameter to estimate force me to use just the first moments in my equation? Am I free to use every moment in my equation?

Thanks in advance.

$\endgroup$
1
  • $\begingroup$ For a Poisson distribution, you can take the average value of the observations as a method of moments estimator of the parameter, or the variance (using $\frac 1n$) or even some weighted average of these two. All of these would be methods of moments estimators, For an exponential distribution, you could take the reciprocal of the average of the observations or the average of the reciprocals of the observations as estimators of the rate parameter (the later uses a negative moment) $\endgroup$
    – Henry
    Jun 4, 2022 at 1:44

1 Answer 1

2
$\begingroup$

You don't even have to use moments. Any function $g(X)$ whose expectation with respect to $P_{\theta}$ uniquely determines $\theta$ will do. By Law of Large Numbers, the average of $g(X_i)$ will converge to that expectation, and you estimate $\theta$ to be the $\theta$ giving this expectation (this only works if the inverse of $\mathbb{E}_{\theta} g(X)$ is continuous in $\theta$). One often uses $g(X)=X^k$, in which case this gives estimates of the k-th mooment of the underlying distribution (from which one deduces the parameters), hence the name.

This answer is shamelessly lifted from https://ocw.mit.edu/courses/18-443-statistics-for-applications-fall-2003/resources/lec3/ where you can find more details.

$\endgroup$
3
  • $\begingroup$ Here is another useful answer I obtained."Theoretical arguments connected to efficiency of estimators justify starting from lower order moments, i.e. one parameter with first moment, two parameters with first two moments and so on. there are exceptions, for example it might happen that the first moment does not depend on any parameter, in this case the equations are built starting from the second moment." Thank you @Max. $\endgroup$
    – Charlie
    Jan 5, 2014 at 20:28
  • $\begingroup$ Link is down, not working. $\endgroup$
    – maenju
    Apr 8, 2022 at 12:23
  • 1
    $\begingroup$ @maenju Try now. $\endgroup$
    – Max
    Apr 8, 2022 at 14:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .