2
$\begingroup$

Let $X_1,X_2,\dots ,X_n$ be a random sample from the Gamma distribution

$$ f(x,\theta)=\theta^2 x e^{-\theta x},\quad x>0$$

To find the Maximum Likelihood Estimator, we define the likelihood function as: $$ L(\theta;x_i)=\prod_{i=1}^nf(x_i;\theta)$$ and we demand $$\frac{\partial \ln[L(\theta;x_i)]}{\partial\theta}=0$$ For this example $$L(\theta;x_i)=\theta^{2n}\cdot \prod_{i=1}^n x_i\cdot e^{-\theta \sum_{i=1}^nx_i}$$ $$\ln[L(\theta;x_i)]=2n\ln(\theta)+n\ln\bigg[\prod_{i=1}^nx_i\bigg]-\theta \sum_{i=1}^nx_i$$ So $$\frac{2n}{\theta}-\sum_{i=1}^n x_i=0\iff \hat{\theta}=\frac{2}{\bar{X}}$$ To find the estimator of $\theta$ using the method of moments $$\bar{X}=E(X)=\mu$$ $$E(X)=\int_0^\infty x f(x)dx=\theta^2\cdot\frac{2}{\theta^3}=\frac{2}{\theta}$$ $$\Rightarrow \tilde{\theta}=\frac{2}{\bar{X}}$$

Is there some kind of relation between the two? Why are the expressions of $\hat{\theta}$ and $\tilde{\theta}$ so similar?

$\endgroup$
4
  • $\begingroup$ There are some errors in your formatting. The likelihood function is a product, so the log-likelihood will include a sum of functions of observations. Anyway, that said, the method of moments you mentioned here uses only information about the first moment, but the MLE approach uses information about all moments in its likelihood function. $\endgroup$ – Vimal Sep 17 '18 at 14:21
  • $\begingroup$ $$\prod_{i=1}^n f(x_i;\theta)=\theta^{2n}\left(\prod_{i=1}^n x_i\right) e^{-\theta \sum_{i=1}^n x_i}\mathbf 1_{x_1,\ldots,x_n>0}$$ $\endgroup$ – StubbornAtom Sep 17 '18 at 14:45
  • $\begingroup$ You're so right. So, the expressions become identical? $\endgroup$ – Andrew Sep 17 '18 at 15:00
  • $\begingroup$ An extra $n$ multiplied by the log of the product in the log-likelihood, but the stationary point for $\theta$ is correct. You would have to confirm that it is a point of maxima however. Also, the parameter space ($\theta>0$) should be mentioned while writing the likelihood. $\endgroup$ – StubbornAtom Sep 17 '18 at 15:14
1
$\begingroup$

For this example

$$L(\theta;x_i)=\theta^{2n}\cdot \prod_{i=1}^n x_i\cdot e^{-\theta \sum_{i=1}^nx_i}$$

This is not right. We have $f(x)=\theta^2 x e^{-\theta x}$ Now we calculate the product for every $x_i$

$$ L(\theta;x_i)=\prod_{i=1}^n \theta^2 x_i\cdot e^{-\theta x_i}=\theta^{2n}\cdot \prod_{i=1}^n x_i\cdot e^{-\theta x_i}$$

You see that there is as yet no sigma sign involved. There is either an sigma sign or a product sign.

At the next step, taking logarithm, there is a mistake. It is right that the $\theta^{2n}$ becomes the summand $2n\cdot \ln(\theta)$. Now we calculate $\ln\left(\prod\limits_{i=1}^n x_i\cdot \large{e^{-\theta x_i}}\right)$

Firstly we use the logarithm rule $\log(a\cdot b)=\log(a)+\log(b)$ to eliminate the product sign.

$$= \sum_{i=1}^n \ln \left( x_i\cdot \large{e^{-\theta x_i}} \right)$$

We use the same rule again for a further simplification.

$$= \sum_{i=1}^n \ln \left( x_i \right) + \sum_{i=1}^n \ln\left( \large{e^{-\theta x_i}} \right)$$

$$= \sum_{i=1}^n \ln \left( x_i \right) -\theta \sum_{i=1}^n x_i$$

With the summnand $2n\cdot \ln (\theta)$ we have

$$\ln \left(L(\theta;x_i)\right)=2n\cdot \ln (\theta) +\sum_{i=1}^n \ln \left( x_i \right) -\theta \sum_{i=1}^n x_i$$

Now the derivative w.r.t. $\theta$ is

$$\frac{2n}{\theta}-\sum_{i=1}^n x_i=0$$

For the rest there are no logarithm rules required.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.