Let $X$ and $Y$ be independent exponential random variables, with

$$f(x\mid\lambda)=\frac{1}{\lambda}\exp{\left(-\frac{x}{\lambda}\right)},\,x>0\,, \qquad f(y\mid\mu)=\frac{1}{\mu}\exp{\left(-\frac{y}{\mu}\right)},\,y>0$$

We observe $Z$ and $W$ with $Z=\min(X,Y)$, and $W=\begin{cases} 1 &,\text{if }Z=X\\ 0 &,\text{if }Z=Y \end{cases}$

I have obtained the joint distribution of $Z$ and $W$, i.e., $$P(Z \leq z, W=0)=\frac{\lambda}{\mu+\lambda}\left[1-\exp{\left(-\left(\frac{1}{\mu}+\frac{1}{\lambda}\right)z\right)}\right]$$

$$P(Z \leq z, W=1)=\frac{\mu}{\mu+\lambda}\left[1-\exp{\left(-\left(\frac{1}{\mu}+\frac{1}{\lambda}\right)z\right)}\right]$$

Now assume that $(Z_i,W_i),i=1,\cdots,n$, are $n$ i.i.d observations. Find the MLEs of $\lambda$ and $\mu$.

(This is the exercise 7.14 of the book Statistical Inference 2nd edition, but no solution given)


Note that $Z$ and $W$ are in fact independent.

For ${z>0\,,\,w\in\{0,1\}}$, you can write the joint distribution of $(Z,W)$ as

$$P(z,w)=\frac{1}{\lambda^w \mu^{1-w}}\exp\left[-\left(\frac{1}{\lambda}+\frac{1}{\mu}\right)z\right]$$

The above formulation is not completely rigorous as it is not a probability (I am searching for a better notation avoiding the dirac delta).

The likelihood function given the sample $(z_1,w_1),\ldots,(z_n,w_n)$ is then

$$L(\lambda,\mu)=\frac{1}{\lambda^{\sum_{i=1}^n w_i}\mu^{n-\sum_{i=1}^n w_i}}\exp\left[-\left(\frac{1}{\lambda}+\frac{1}{\mu}\right)\sum_{i=1}^n z_i\right]$$

Log-likelihood is

$$\ell(\lambda,\mu)=-\sum_{i=1}^n w_i\ln\lambda-\left(n-\sum_{i=1}^n w_i\right)\ln\mu--\left(\frac{1}{\lambda}+\frac{1}{\mu}\right)\sum_{i=1}^n z_i$$

For $0<\bar w<1$, solving for the stationary points of $\ell(\lambda,\mu)$ yields $$\hat\lambda=\frac{\sum_{i=1}^n z_i}{\sum_{i=1}^n w_i}=\frac{\bar z}{\bar w}\qquad,\qquad \hat\mu=\frac{\sum_{i=1}^n z_i}{n-\sum_{i=1}^n w_i}=\frac{\bar z}{1-\bar w}$$

So assuming $0<\bar w<1$, the unique MLE of $(\lambda,\mu)$ is $(\hat\lambda,\hat\mu)$.

But when $\bar w\in\{0,1\}$, the MLE does not exist.

  • $\begingroup$ I still have two questions. (Q1) How to solve for the stationary points of $\ell(\lambda,\mu)$? (Q2) Why do we need to discuss the value range of $\bar{w}$? (In addition) Shouldn't the $\mu$ in your Log-likelihood is $\ln\mu$? $\endgroup$ – Tim Xu Mar 21 at 12:08
  • $\begingroup$ You are right about the log-likelihood. I will edit. $\endgroup$ – StubbornAtom Mar 21 at 12:18
  • $\begingroup$ @TimXu I just differentiated $\ell$ once wrt $\lambda$ and $\mu$ and set them equal to zero (for multiple parameters, one should ideally check whether the Hessian matrix of the log-likelihood is negative definite at $(\hat\lambda,\hat\mu)$, provided the derivatives exist). And you can see from the expression for $\hat\lambda,\hat\mu$ that they are not defined at $\bar w=0,1$. Hence the range distinction. $\endgroup$ – StubbornAtom Mar 21 at 12:35

The $W=0$ and the $W=1$ case can be combined by writing $\lambda^{1-W}\mu^W$.

First differentiating w.r.t. $z$ and forming the product the likelihood can be written as $\prod_i^n{\frac{1}{\lambda^{w_i}\mu^{1-w_i}}}e^{-(\frac{1}{\lambda}+\frac{1}{\mu})z_i}$. Taking logs gives the log likelihood $=-\sum_i^n(w_i\ln{\lambda}+(1-w_i)\ln{\mu}+(\frac{1}{\lambda}+\frac{1}{\mu})z_i)$.

Maximising wrt $\lambda$ and $\mu$ gives $\lambda=\frac{\bar{z}}{\bar{w}}$ and $\mu=\frac{\bar{z}}{(1-\bar{w})}$

Note if $\lambda=\mu$ then $\bar{w}\approx\frac{1}{2}$ so the estimates for $\lambda$ and $\mu$ become equal.

  • $\begingroup$ Denote the log likelihood by $\log L$. First differentiating w.r.t. $\lambda$ and $\mu$, gives $\hat{\lambda}=\frac{\bar{z}}{\bar{w}}$, $\hat{\mu}=\frac{\bar{z}}{1-\bar{w}}$. To make sure they are the maximizers of the log likelihood. I calculate their second-order differentials w.r.t. $\lambda$ and $\mu$, and get that $\frac{\partial^2 \log L}{\partial \lambda^2}=-\sum_{i=1}^n(-\frac{w_i}{\lambda^2}+2\frac{z_i}{\lambda^3})$ and $\frac{\partial^2 \log L}{\partial \mu^2}=-\sum_{i=1}^n(-\frac{1-w_i}{\mu^2}+2\frac{z_i}{\mu^3})$. How do you know these two second-order differentials are negative? $\endgroup$ – Tim Xu Mar 21 at 11:44
  • $\begingroup$ Direct substitution. for instance $\frac{\partial^2 \log{L}}{\partial \lambda^2}=-\frac{\bar{w}^3}{\bar{z}^2}$ $\endgroup$ – user121049 Mar 21 at 14:09
  • $\begingroup$ You missed an $n$. It is $\frac{\partial ^2 \log{L}}{\partial \lambda^2} |_{\lambda=\frac{\bar z}{\bar w}}= -\sum_{i=1}^n (-\frac{w_i}{\lambda^2}+2\frac{z_i}{\lambda^3}) |_{\lambda=\frac{\bar z}{\bar w}}= (\frac{\sum_{i=1}^n w_i}{\lambda^2}-2\frac{\sum_{i=1}^n z_i}{\lambda^3}) |_{\lambda=\frac{\bar z}{\bar w}}= (\frac{n \bar w}{\lambda^2}-2\frac{n \bar z}{\lambda^3}) |_{\lambda=\frac{\bar z}{\bar w}}= \frac{n \bar w^3}{\bar z^2}-2\frac{n \bar z \bar w^3}{\bar z^3}= \frac{n \bar w^3}{\bar z^2}-2\frac{n \bar w^3}{\bar z^2}=-\frac{n \bar w^3}{\bar z^2}$ $\endgroup$ – Tim Xu Mar 22 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.