I am having a bit of a problem with examining the properties of a maximum likelihood estimator. I feel like I am missing something simple, but I have been unable to find someone doing an example quite like this and keep getting a bit lost.

So, basically, I have this random sample ($X_1,\ldots,X_n$) from an exponential distribution with parameter $\theta$.

EDIT: I forgot to mention, I am using this form of the exponential pdf:


That is, $\lambda=\frac{1}{\theta}$. Just realized I should specify before people get confused. The reasons for using this variant of the pdf are sort of convoluted, and it unfortunately isn't feasible to change at the moment.

Finding the MLE and its properties are, of course, trivial. The MLE is simply the sample mean ($\overline{X}_n$). However, it is not this MLE I am interested in.

I am interested in a parallel random sample ($Z_1,\ldots,Z_n$), where $Z_i=\mathbb{I}(X_i>1)$. Again, finding the MLE isn't terribly difficult, due to the invariance property. That is, if $\hat{\theta}_x$ is the MLE of $\theta$ with respect to $X$, then $\tau({\hat{\theta}_x})$ is the MLE of $\tau(\theta)$. So,

$\tau(\theta)=P(X>1)= exp(-\frac{1}{\theta})$

$\tau({\hat{\theta}_x}) = exp(-\frac{1}{\overline{X}_n})$

gives me the MLE of $\theta$ with respect to $Z$. (I think… does this look right to everyone?)

However, this is where I get stuck. I am trying to investigate the properties of this MLE. Specifically, its bias/variance/MLE/asymptotic distribution.

How do I evaluate $E[e^{-\frac{1}{{\overline{X}}_n}}]$? I don't even know what distribution to take this expectation in relation to via integration/MGFs. Is it the distribution of $X_i$? Of $X_1,\ldots,X_n$? $Z_i$? $Z_1,\ldots,Z_n$?

Searching around, there was a similar problem encountered by a poster here: Maximum likelihood estimator of $P(X < y)$ for fixed $y$

However, the final solution isn't posted, and I find the advice given to be confusingly phrased, so it isn't terribly helpful.

Anyway, anyone have some pointers for me? I can't see an easy solution.

  • $\begingroup$ Updated post to reflect new developments $\endgroup$ – Ryan Simmons Mar 29 '14 at 11:45
  • $\begingroup$ You need to be clear about what you mean by "MLE of $\theta$ with respect to $Z$": do you mean, given only the data $(Z_1, \ldots, Z_n)$, what is the MLE of $\theta$? In that case, the answer cannot be the expression you have, because that is a function of $(X_1, \ldots, X_n)$. $\endgroup$ – heropup Apr 3 '14 at 13:57

Suppose we are given only $\boldsymbol Z = (Z_1, \ldots, Z_n)$, and we wish to find the MLE for the parameter $\theta$. Then each $Z_i \sim {\rm Bernoulli}(p_i)$, where $p_i = \Pr[Z_i = 1] = \Pr[X_i > 1] = e^{-1/\theta}.$ Consequently, the likelihood for $\theta$ given $\boldsymbol Z$ is $$L(\theta \mid \boldsymbol Z) = \prod_{i=1}^n p_i^{Z_i} (1-p_i)^{1-Z_i} = e^{-n\bar Z/\theta} (1-e^{-1/\theta})^{n(1-\bar Z)}.$$ The log-likelihood is $$\ell(\theta \mid \boldsymbol Z) = -\frac{n \bar Z}{\theta} + n(1-\bar Z) \log(1-e^{-1/\theta}).$$ Taking the derivative with respect to $\theta$ and finding the critical points gives $$0 = \frac{n(\bar Z - e^{-1/\theta})}{\theta^2 (1 - e^{-1/\theta})},$$ or $$\hat \theta_Z = - \frac{1}{\log \bar Z}, \quad 0 < \bar Z < 1.$$ Note that if $\bar Z = 0$, then $\hat \theta_Z = 0$. More problematically, if $\bar Z = 1$, then $\hat \theta_Z = + \infty$. Equivalently, we could have immediately obtained this result from the familiar MLE of an IID Bernoulli sample, which is $\hat p = \bar Z$, and the transformation $$\hat \theta_Z = \tau(\hat p) = -\frac{1}{\log \hat p}.$$ What is its bias? It is obviously infinite: since $n \bar Z \sim {\rm Binomial}(n, p = e^{-1/\theta})$, there is a positive probability that $\bar Z = 1$ for any $\theta > 0$. Therefore, ${\rm E}[\hat \theta_Z]$ cannot be finite. This also means the variance of the estimator is undefined.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.