2
$\begingroup$

Hello I am trying to check if the Method of Moments and Maximum Likelihood Estimators for parameter $\theta$ from a sample with population density $$f(x;\theta) = \frac 2 \theta x e^{\frac {-x^2}{\theta}} $$ for $x \geq 0$, $\theta > 0$ with $\theta$ being unknown.

Taking the first moment of this function I found the Method of Moments estimator to be $\hat{\theta}_1 = \frac{4(\bar{X}^2)}{\pi}$ and solving for the Maximum Likelihood Estimator the Estimator to be $\hat{\theta}_2 = 2\bar Y$ where $Y$ is just square of the Sample $X_i$, i.e. $Y = X_i^2$.

Steps in Solving for Method of Moments: I took the first moment, i.e.

$M_1 = E[x] = $$\int_0^\infty{\frac 2 \theta x^2 e^{\frac {-x^2}{\theta}}dx}$

Solving this integral with $u$ substitution with $u = \frac{-x}{2}, du = \frac{-1}{2}, v = e^\frac{x^2}{\theta}, dv = -2xe^\frac{-x^2}{\theta}$

$\int_0^\infty{\frac 2 \theta x^2 e^{\frac {-x^2}{\theta}}dx} = [-\frac{xe^\frac{-x^2}{\theta}}{2\theta} - \frac{\sqrt{\pi \theta}}{4}]^\infty_0 = \frac{\sqrt{\pi} {\sqrt{\theta}}}{2}$

So that $E[x] = \bar{x} = \frac{\sqrt{\pi} {\sqrt{\theta}}}{2}$ gives the Method of Moments Estimator $\hat{\theta_1} = \frac{4(\bar{X}^2)}{\pi}$

Steps in Solving for Maximum Likelihood:

$lnL(\theta)=(\prod_{i=1}^n\frac 2 \theta x e^{\frac {-x^2}{\theta}}) = -n ln((2\theta)) + \sum_{i=1}^nx_i - \frac {1} {\theta} \sum_{i=1}^nx^2_i$

$\frac {dlnL(\theta)}{d\theta} = \frac{-n}{2\theta} + \frac{1}{\theta^2} \sum_{i=1}^nx^2_i$

Setting $\frac {dL(\theta)}{d\theta} = 0$, I found the Maximum Likelihood Estimator $\hat{\theta_2}$ to be $\hat{\theta_2} = \frac{2\sum_{i=1}^nx^2_i}{n}$ , so that if $Y = X_i^2$ then $\hat{\theta_2} = 2\bar{Y}$.

I am trying to check if these estimators for $\theta$ from this density function are unbiased and/or consistent but am lost on how to go about doing so, any help would be much appreciated.

$\endgroup$
5
  • $\begingroup$ Unfortunately, both estimators are wrong. You can edit your question and add the finding of these estimators so that we can find mistakes. $\endgroup$
    – NCh
    May 27, 2017 at 17:56
  • $\begingroup$ Okay so I have edited my post by adding the steps I took in finding both estimators, did I make a mistake in calculating them? $\endgroup$
    – Jvector900
    May 29, 2017 at 7:31
  • $\begingroup$ If $v = \frac{-e^\frac{x^2}{\theta}}{2x}$, then $dv \neq e^\frac{-x^2}{\theta} dx$. $\hat{\theta_2}$ is right. $\endgroup$
    – NCh
    May 29, 2017 at 11:39
  • $\begingroup$ I see thank you for pointing out my mistake in the integration of the first moment, it is not as simple as I thought it would be and involves a Gaussian integral.Evaluating the integral with a computer software the answer to E[x] is $\frac {\sqrt{\pi \theta}}{2}$, which would imply $\hat{\theta_1}$ is $\hat{\theta_1} = \frac{4 \bar{X^2}}{\pi}$, is this the correct estimator? $\endgroup$
    – Jvector900
    May 29, 2017 at 20:19
  • $\begingroup$ Yes. And the estimator is $\hat{\theta_1} = \frac{4 (\bar{X})^2}{\pi}$, not $\frac{4 \bar{X^2}}{\pi}$. $\endgroup$
    – NCh
    May 30, 2017 at 0:38

1 Answer 1

1
$\begingroup$

Your calculation of $\hat \theta_2$ is still not right. You should have $$\mathcal L(\theta \mid \boldsymbol x) = 2^n \theta^{-n} \left ( \prod_{i=1}^n x_i \right) \exp \left( -\frac{1}{\theta} \sum_{i=1}^n x_i^2 \right) \propto \theta^{-n} \exp \left( - \frac{n \overline{x^2}}{\theta} \right),$$ thus your log-likelihood is $$\ell (\theta \mid \boldsymbol x) = -n \log \theta - \frac{n \overline{x^2}}{\theta},$$ and locating the critical points gives $$0 = \frac{\partial \ell}{\partial \theta} = -\frac{n}{\theta} + \frac{n \overline{x^2}}{\theta^2} = n \left( \frac{\overline{x^2} - \theta}{\theta^2} \right),$$ hence $\hat \theta_2 = \overline{x^2} = \frac{1}{n} \sum_{i=1}^n x_i^2.$ There is no additional factor of $2$. Notice how I remove all factors of $\mathcal L$ that are not functions of $\theta$, which simplifies all subsequent calculations (and avoids the computational error you made with the additional factor of $2$).

To compute the bias of these estimators, it suffices to use the basic properties: First, observe $$\operatorname{E}[X] = \frac{\sqrt{\pi \theta}}{2}$$ as you wrote. You may verify that $$\operatorname{E}[X^2] = \theta.$$ Now we see that $$\operatorname{E}[\hat\theta_2] = \operatorname{E}\left[\frac{1}{n} \sum_{i=1}^n X_i^2\right] = \frac{1}{n} \sum_{i=1}^n \operatorname{E}[X_i^2] = \frac{1}{n} \cdot n \theta = \theta,$$ so $\hat\theta_2$ is unbiased. This is the most immediately obvious calculation which is why we started with it. As for $\hat \theta_1$, we must be careful to write $$\operatorname{E}[\hat\theta_1] = \frac{4}{\pi} \operatorname{E}\,\left[\left(\frac{1}{n} \sum_{i=1}^n X_i\right)^2\right] = \frac{4}{\pi n^2} \sum_{i=1}^n \sum_{j=1}^n \operatorname{E}[X_i X_j].$$ Note we have not yet used independence of the sample, only the linearity of expectation. When $i \ne j$, $X_i$ and $X_j$ are independent, and $$\operatorname{E}[X_i X_j] = \operatorname{E}[X_i]\operatorname{E}[X_j] = \frac{\pi}{4}\theta.$$ But when $i = j$, this is not the case and we have $$\operatorname{E}[X_i X_j] = \operatorname{E}[X_i^2] = \theta.$$ Since the first case occurs $n(n-1)$ times in the double sum, and the second occurs $n$ times, we get $$\operatorname{E}[\hat\theta_1] = \frac{4}{\pi n^2} \left( n(n-1) \frac{\pi}{4}\theta + n \theta \right) =\left( 1 + \frac{4 - \pi}{\pi n} \right) \theta .$$ This of course proves $\hat\theta_1$ is biased. Is it asymptotically biased or unbiased?

As for the consistency of the estimator, you must show using similar methods that the variance decreases with increasing sample size $n$. To do this, you must compute $\operatorname{E}[\hat\theta_1^2]$ and $\operatorname{E}[\hat\theta_2^2]$. This is left as an exercise.


The second moment of $\hat \theta_1$ is $$\operatorname{E}[\hat\theta_1^2] = \frac{4^2}{\pi^2} \operatorname{E}\,\left[\left(\frac{1}{n}\sum_{i=1}^n X_i\right)^4\right].$$ Use the same technique as for the first moment: $$\left(\sum_{i=1}^n X_i\right)^4 = \sum_{g=1}^n \sum_{h=1}^n \sum_{i=1}^n \sum_{j=1}^n X_g X_h X_i X_j.$$ How many of these summands correspond to all distinct indices? How many correspond to exactly two equal? How many correspond to two equal pairs? How many correspond to exactly three equal? How many correspond to all four equal? What is the expectation of a general term in each case?

$\endgroup$
9
  • $\begingroup$ Thank you for correcting my mistake in computing for $\hat{\theta_2}$ and showing that both estimators are biased. I am trying to compute for the variance of both estimators now and believe $V[\hat{\theta_2}] = E[\hat{\theta_2}^2] - (E[\hat{\theta_2}])^2 = \frac{1}{n^2} E[\sum_{n = 1}^{\infty} X^4_i] - \theta^2 = \frac{2 \theta^2}{n} - \theta^2$ is this correct? Also how do I compute for $E[\hat{\theta_1}^2]?$ $\endgroup$
    – Jvector900
    May 31, 2017 at 8:12
  • $\begingroup$ @Jvector900 $\hat\theta_1$ is biased. $\hat\theta_2$ is unbiased. Your calculation of $\operatorname{Var}[\hat\theta_2]$ is incorrect, since $$\left(\sum_{i=1}^n X_i^2\right)^2 \ne \sum_{i=1}^n X_i^4.$$ $\endgroup$
    – heropup
    May 31, 2017 at 15:45
  • $\begingroup$ I solved for the variance of $\hat{\theta_2}$ by transforming the original distribution into an exponential by letting $u = x^2$ and $du = 2x$ and noting that this reduces the distribution into an exponential distribution with variance $\theta^2$ so the maximum likelihood estimator is just $\bar{u}$ and hence has variance is $\frac{\theta^2}{n}$. I still don't know how to calculate for the variance of the Method of Moments estimator $\hat{\theta_1} = \frac{4 (\bar{X})^2}{\pi}$, how do I solve for $\hat{\theta_1}$? $\endgroup$
    – Jvector900
    Jun 2, 2017 at 17:57
  • $\begingroup$ So there are 4 different cases, each corresponding to the 1st 2nd 3rd or 4th moment? 1 case being n different sums when g=h=i-=j, n(n - 1) where 3 of the indices are equal involving the 3rd moment, n(n - 1)(n - 2) involving the 2nd, and n(n - 1)(n -2)(n -3) involving the 1st moment when all the indices are unique. Do I need to divide by a number for each case to avoid over counting? $\endgroup$
    – Jvector900
    Jun 3, 2017 at 21:13
  • $\begingroup$ There are $n(n-1)(n-2)(n-3)$ cases where all indices are distinct; e.g. $(g,h,i,j) = (1,2,3,4)$. There are $6n(n-1)(n-2)$ cases where exactly two indices are equal; e.g. $(1,2,1,3)$. There are $3n(n-1)$ cases where two pairs of two indices are equal; e.g. $(1,2,2,1)$. There are $4n(n-1)$ cases where exactly three indices are equal; e.g. $(1,1,2,1)$. There are $n$ cases where all four are equal. Then $$n(n-1)(n-2)(n-3) + 6n(n-1)(n-2) + 3n(n-1) + 4n(n-1) + n = n^4.$$ Each case has a different expectation. $\endgroup$
    – heropup
    Jun 4, 2017 at 3:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .