3
$\begingroup$

Given the $X_i\sim \text{exp}({\theta})$ and $Y_i\sim \text{exp}(\frac{1}{\theta})$, where $X_i$ and $Y_i$ are indpendent, with the same $\theta>0$. I have to find the MLE and its distribution.

I find the simultaneous distribution to be: $$f_{X_i,Y_i}(x,y)=e^{-\left(\frac{x_i}{\theta }+\theta y_i\right)}1_{(0,\infty )}(x_i)1_{(0,\infty )}(y_i),$$ and the Fisherinformation: $$i_n(\theta)=\frac{2 n}{\theta ^2}$$

As it's convex, the maximum of the loglikelihood must be a minima, so I find the MLE: $$\hat{\theta}=\left(\frac{X_{\bullet }}{Y_{\bullet }}\right)^{0.5},$$ where $X_{\bullet }=\sum _{i=1}^n x_i$ and $Y_{\bullet }=\sum _{i=1}^n y_i$. So the asymptotic distribution is: $$\hat{\theta}\sim\mathcal{N}\left(\theta ,\frac{\theta^2}{2n^2}\right),$$ as $\hat{\theta}\approx\mathcal{N}\left(\theta,\frac{1}{ni_n(\theta)}\right),$ where $i_\theta$ is the Fisherinformation.

I am in doubt whether this is the right procedure.


EDIT

It is the right procedure - my professor provided the correct solution.

$\endgroup$
3
$\begingroup$

For a sample of $n$ observations on $(X_i,Y_i)$, the likelihood function given $(x_1,y_1),\ldots,(x_n,y_n)$ is

\begin{align} L(\theta)&=\prod_{i=1}^n e^{-\left(x_i/\theta+\theta y_i\right)}\mathbf1_{x_i>0,y_i>0} \\&=\exp\left[{-\sum_{i=1}^n\left(\frac{x_i}{\theta}+\theta y_i\right)}\right]\mathbf1_{x_1,\ldots,x_n,y_1,\ldots,y_n>0} \\&=\exp\left[-\frac{n\bar x}{\theta}-\theta n\bar y\right]\mathbf1_{x_{(1)},y_{(1)}>0}\quad,\,\theta>0 \end{align}

This, as you say, yields the MLE $$\hat\theta(\mathbf X,\mathbf Y)=\sqrt{\frac{\overline X}{\overline Y}}$$, where $\overline X,\overline Y$ are the sample means.

Now $X_i$'s are i.i.d $\mathsf{Exp}$ with mean $\theta$, and $Y_i$'s are i.i.d $\mathsf{Exp}$ with mean $1/\theta$, where $X_i,Y_i$ are independent.

Or, $$\frac{2}{\theta}X_i\stackrel{\text{ i.i.d }}\sim\chi^2_2\quad,\text{ independent of }\quad2\theta Y_i\stackrel{\text{ i.i.d }}\sim\chi^2_2$$

Therefore summing over the observations, we have

$$\frac{2}{\theta}n\overline X\sim\chi^2_{2n}\quad,\text{ independent of }\quad 2\theta n\overline Y\sim\chi^2_{2n}$$

Taking ratio of the statistics gives $$\frac{\overline X}{\theta^2 \overline Y}\sim F_{2n,2n}$$

So the (exact) distribution of the MLE $\hat\theta$ can be expressed as $$\frac{\hat\theta}{\theta}\stackrel{d}{=} \sqrt F\quad,\text{ where }F\sim F_{2n,2n}$$

I don't think the approach with asymptotic distribution is correct.


Since by (weak) law of large numbers $\frac{1}{n}\sum\limits_{i=1}^n X_i\stackrel{P}\longrightarrow E(X_1)=\theta$ and $\frac{1}{n}\sum\limits_{i=1}^n Y_i\stackrel{P}\longrightarrow E(Y_1)=\frac{1}{\theta}$,

$$\hat\theta^2=\frac{\frac{1}{n}\sum\limits_{i=1}^n X_i}{\frac{1}{n}\sum\limits_{i=1}^n Y_i}\stackrel{P}\longrightarrow \frac{E(X_1)}{E(Y_1)}=\theta^2$$

And by continuous mapping theorem, $$\hat\theta \stackrel{P}\longrightarrow \theta$$

As a result, asymptotic distribution of the MLE $\hat\theta$ is degenerate at $\theta$.

Also see this relevant post.

$\endgroup$
  • $\begingroup$ Thank you, @StubbornAtom. Am I correct if I say, that the asymptotic distribution for the $\hat{\theta}$ is correct? Have you any advice in regards to knowing when the asymptotic approach is not best? $\endgroup$ – Frederik Apr 20 at 2:54
  • $\begingroup$ @Frederik Why look for asymptotic distribution when exact distribution is easily available? I don't understand how you obtained the asymptotic distribution in your post. The MLE has an asymptotic normal distribution (under some conditions) typically when there is a single sample of observations. Here you have two (independent) samples. $\endgroup$ – StubbornAtom Apr 20 at 7:54
  • $\begingroup$ I understand your concern, however, the reason I am looking for the asymptotic distribution is that I am solving an old exam set. In this set, they want me to find MLE, as I have, and the asymptotic distribution hereof - even though finding the exact distribution would be better. I have found the asymptotic distribution with formula: $$\hat{\theta}\approx\mathcal{N}\left(\theta,\hat{\text{se}}^2\right)=\mathcal{N}\left(\theta,\frac{1}{I_n(\theta)}\right),$$ from "All of Statistics" on p. 129. $\endgroup$ – Frederik Apr 20 at 8:18
  • 1
    $\begingroup$ @Frederik Have a look at my edit. $\endgroup$ – StubbornAtom Apr 20 at 14:23
  • $\begingroup$ Thank you, @StubbornAtom. I will try to look into it, as I'm not that familiar with the understanding of the way that MLE can degenerate. $\endgroup$ – Frederik Apr 21 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.