Let $X_1,...,X_n$ be a random sample from the pdf $$f(x|\theta) = \theta x^{\theta-1} , 0 \leq x \leq 1, \theta >0.$$

I found the Maximum-likelihood estimator of $\theta$ is $$\hat{\theta} = \frac{-n}{\sum_{i=1}^N \ln(X_i)}.$$ Can anyone confirm that this is right?

Then, I want to determine whether $\hat{\theta}$ has bias. My approach is to calculate ${\bf E}[\hat{\theta}] = {\bf E}\left[\frac{-n}{\sum_{i=1}^N \ln(X_i)}\right]$...Then I am stuck. Could someone help me with this?


You got the right answer for the MLE. To find the expectation of $\hat{\theta}$, it will help to first find the distribution of the transformation $Y=-log(X_i)$. It's a well known distribution, and so will be $-\sum log(X_i)$ as well as $-1/\sum log(X_i)$.

You have already found that the distribution of $Y_i=-log(X_i)$ is exponential($1/\theta$).

According to section 4.2 of Pitman, "Probability", the sum of n iid exponential($1/\theta$) distributions is gamma(n, $1/\theta$). Thus, Z =$\sum Y_i=-\sum log(X_i)$ is distributed gamma(n,$1/\theta$) where the pdf of Z is

$\dfrac{\theta^n}{\Gamma(n)}z^{n-1}e^{-z\theta}$ with mean $E(Z)=n/\theta$.

Notice that since a pdf must integrate to 1, it is straightforward to show

$\int_0^{\infty}z^{n-1}e^{-z\theta}dz=\dfrac{\Gamma(n)}{\theta^n}$. Call this result 1.

This will be useful in the following step to find $E(1/Z)$.

$E(1/Z) = \int_0^{\infty}1/z\dfrac{\theta^n}{\Gamma(n)}z^{n-1}e^{-z\theta}dz$

$= \dfrac{\theta^n}{\Gamma(n)} \int_0^{\infty}z^{(n-1)-1}e^{-z\theta}$

$= \dfrac{\theta^n}{\Gamma(n)}\dfrac{\Gamma(n-1)}{\theta^{n-1}}$ by result 1.


Now, according to Casella and Berger, Statistical Inference pg 99, a useful property of the gamma function is that

$\Gamma(\alpha+1)=\alpha\Gamma(\alpha)$, thus

$\Gamma(n)=\Gamma((n-1)+1) = (n-1)\Gamma(n-1)$ and

$E(1/Z) = \dfrac{\theta\Gamma(n-1)}{\Gamma(n)} =\dfrac{\theta\Gamma(n-1)}{(n-1)\Gamma(n-1)}= \theta/(n-1) $

It's now straightforward to find $E(n/Z) = n*E(1/Z) = n\theta/(n-1) $.

  • $\begingroup$ I found that $log(X_i)$ has an exponential distribution with mean $1/\theta$. How does this help? $\endgroup$ – afsdf dfsaf May 13 '14 at 14:51
  • $\begingroup$ Do you know what the distribution is of a sum of exponential distributions? $\endgroup$ – jsk May 13 '14 at 14:54
  • $\begingroup$ is it going to be gamma distribution? $\endgroup$ – afsdf dfsaf May 13 '14 at 14:58
  • 1
    $\begingroup$ whoa, wait a sec. You have to pay attention to the details. $E(1/X)$ DOES NOT EQUAL $1/E(X)$!!!! $\endgroup$ – jsk May 14 '14 at 4:23
  • 1
    $\begingroup$ YOU CAN USE THE GAMMA APPROACH, but you CANNOT claim that $ E(-1/\sum log(X_i)) = 1/E(-\sum log(X_i))$ THIS IS NOT TRUE BECAUSE OF JENSEN'S INEQUALITY $\endgroup$ – jsk May 14 '14 at 4:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.