3
$\begingroup$

Given the $X_i\sim \text{exp}({\theta})$ and $Y_i\sim \text{exp}(\frac{1}{\theta})$, where $X_i$ and $Y_i$ are indpendent, with the same $\theta>0$. I have to find the MLE and its distribution.

I find the simultaneous distribution to be: $$f_{X_i,Y_i}(x,y)=e^{-\left(\frac{x_i}{\theta }+\theta y_i\right)}1_{(0,\infty )}(x_i)1_{(0,\infty )}(y_i),$$ and the Fisherinformation: $$i_n(\theta)=\frac{2 n}{\theta ^2}$$

As it's convex, the maximum of the loglikelihood must be a minima, so I find the MLE: $$\hat{\theta}=\left(\frac{X_{\bullet }}{Y_{\bullet }}\right)^{0.5},$$ where $X_{\bullet }=\sum _{i=1}^n x_i$ and $Y_{\bullet }=\sum _{i=1}^n y_i$. So the asymptotic distribution is: $$\hat{\theta}\sim\mathcal{N}\left(\theta ,\frac{\theta^2}{2n^2}\right),$$ as $\hat{\theta}\approx\mathcal{N}\left(\theta,\frac{1}{ni_n(\theta)}\right),$ where $i_\theta$ is the Fisherinformation.

I am in doubt whether this is the right procedure.


EDIT

It is the right procedure - my professor provided the correct solution.

$\endgroup$
3
$\begingroup$

For a sample of $n$ observations on $(X_i,Y_i)$, the likelihood function given $(x_1,y_1),\ldots,(x_n,y_n)$ is

\begin{align} L(\theta)&=\prod_{i=1}^n e^{-\left(x_i/\theta+\theta y_i\right)}\mathbf1_{x_i>0,y_i>0} \\&=\exp\left[{-\sum_{i=1}^n\left(\frac{x_i}{\theta}+\theta y_i\right)}\right]\mathbf1_{x_1,\ldots,x_n,y_1,\ldots,y_n>0} \\&=\exp\left[-\frac{n\bar x}{\theta}-\theta n\bar y\right]\mathbf1_{x_{(1)},y_{(1)}>0}\quad,\,\theta>0 \end{align}

This, as you say, yields the MLE $$\hat\theta(\mathbf X,\mathbf Y)=\sqrt{\frac{\overline X}{\overline Y}}$$, where $\overline X,\overline Y$ are the sample means.

Now $X_i$'s are i.i.d $\mathsf{Exp}$ with mean $\theta$, and $Y_i$'s are i.i.d $\mathsf{Exp}$ with mean $1/\theta$, where $X_i,Y_i$ are independent.

Or, $$\frac{2}{\theta}X_i\stackrel{\text{ i.i.d }}\sim\chi^2_2\quad,\text{ independent of }\quad2\theta Y_i\stackrel{\text{ i.i.d }}\sim\chi^2_2$$

Therefore summing over the observations, we have

$$\frac{2}{\theta}n\overline X\sim\chi^2_{2n}\quad,\text{ independent of }\quad 2\theta n\overline Y\sim\chi^2_{2n}$$

Taking ratio of the statistics gives $$\frac{\overline X}{\theta^2 \overline Y}\sim F_{2n,2n}$$

So the (exact) distribution of the MLE $\hat\theta$ can be expressed as $$\frac{\hat\theta}{\theta}\stackrel{d}{=} \sqrt F\quad,\text{ where }F\sim F_{2n,2n}$$


Since by (weak) law of large numbers $\frac{1}{n}\sum\limits_{i=1}^n X_i\stackrel{P}\longrightarrow E(X_1)=\theta$ and $\frac{1}{n}\sum\limits_{i=1}^n Y_i\stackrel{P}\longrightarrow E(Y_1)=\frac{1}{\theta}$,

$$\hat\theta^2=\frac{\frac{1}{n}\sum\limits_{i=1}^n X_i}{\frac{1}{n}\sum\limits_{i=1}^n Y_i}\stackrel{P}\longrightarrow \frac{E(X_1)}{E(Y_1)}=\theta^2$$

And by continuous mapping theorem, $$\hat\theta \stackrel{P}\longrightarrow \theta$$

This further implies convergence in distribution. So as one might expect, the asymptotic distribution of $\hat\theta$ is degenerate at $\theta$.


Asymptotic normality of MLE is a general result which holds under some smoothness assumptions of the population density. These so called regularity conditions are sufficient conditions for verifying asymptotic normality. If these conditions hold, then indeed a non-degenerate limiting distribution of MLE is given by

$$\sqrt n\left(\hat\theta-\theta\right)\stackrel{L}\longrightarrow N\left(0,\frac1{I(\theta)}\right)\,,$$

where $I(\theta)$ is the Fisher information contained in a single observation.

For a direct proof of asymptotic normality in this particular problem, see this answer.

Alternatively, we can use the multivariate central limit theorem combined with the delta method to reach the same answer.

By CLT we have

$$\sqrt n\left(\left(\overline X,\overline Y\right)-\left(\theta,\frac1{\theta}\right)\right)\stackrel{L}\longrightarrow N\left(\mathbf 0,\Sigma\right)\,,$$

where $$\Sigma=\begin{pmatrix}\theta^2 & 0 \\ 0 & \frac1{\theta^2}\end{pmatrix}$$

By the delta-method, the following holds for some function $g$ :

$$\sqrt n\left(g\left(\overline X,\overline Y\right)-g\left(\theta,\frac1{\theta}\right)\right)\stackrel{L}\longrightarrow N\left( 0, \nabla g\left(\theta,\frac1{\theta}\right)^T \cdot \Sigma \cdot \nabla g\left(\theta,\frac1{\theta}\right)\right)$$

Taking $g(x,y)=\sqrt{\frac xy}$ for $x,y>0$, this reduces to

$$\sqrt n\left(\hat\theta-\theta\right)\stackrel{L}\longrightarrow N\left(0,\frac{\theta^2}{2}\right)$$

$\endgroup$
  • $\begingroup$ Thank you, @StubbornAtom. Am I correct if I say, that the asymptotic distribution for the $\hat{\theta}$ is correct? Have you any advice in regards to knowing when the asymptotic approach is not best? $\endgroup$ – Frederik Apr 20 '19 at 2:54
  • $\begingroup$ I understand your concern, however, the reason I am looking for the asymptotic distribution is that I am solving an old exam set. In this set, they want me to find MLE, as I have, and the asymptotic distribution hereof - even though finding the exact distribution would be better. I have found the asymptotic distribution with formula: $$\hat{\theta}\approx\mathcal{N}\left(\theta,\hat{\text{se}}^2\right)=\mathcal{N}\left(\theta,\frac{1}{I_n(\theta)}\right),$$ from "All of Statistics" on p. 129. $\endgroup$ – Frederik Apr 20 '19 at 8:18
  • 1
    $\begingroup$ @Frederik Have a look at my edit. $\endgroup$ – StubbornAtom Apr 20 '19 at 14:23
  • $\begingroup$ Thank you, @StubbornAtom. I will try to look into it, as I'm not that familiar with the understanding of the way that MLE can degenerate. $\endgroup$ – Frederik Apr 21 '19 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.