# How to find the MLE of these parameters given distribution?

Let $$X$$ and $$Y$$ be independent exponential random variables, with

$$f(x\mid\lambda)=\frac{1}{\lambda}\exp{\left(-\frac{x}{\lambda}\right)},\,x>0\,, \qquad f(y\mid\mu)=\frac{1}{\mu}\exp{\left(-\frac{y}{\mu}\right)},\,y>0$$

We observe $$Z$$ and $$W$$ with $$Z=\min(X,Y)$$, and $$W=\begin{cases} 1 &,\text{if }Z=X\\ 0 &,\text{if }Z=Y \end{cases}$$

I have obtained the joint distribution of $$Z$$ and $$W$$, i.e., $$P(Z \leq z, W=0)=\frac{\lambda}{\mu+\lambda}\left[1-\exp{\left(-\left(\frac{1}{\mu}+\frac{1}{\lambda}\right)z\right)}\right]$$

$$P(Z \leq z, W=1)=\frac{\mu}{\mu+\lambda}\left[1-\exp{\left(-\left(\frac{1}{\mu}+\frac{1}{\lambda}\right)z\right)}\right]$$

Now assume that $$(Z_i,W_i),i=1,\cdots,n$$, are $$n$$ i.i.d observations. Find the MLEs of $$\lambda$$ and $$\mu$$.

(This is the exercise 7.14 of the book Statistical Inference 2nd edition, but no solution given)

Note that $$Z$$ and $$W$$ are in fact independent.

For $${z>0\,,\,w\in\{0,1\}}$$, you can write the joint distribution of $$(Z,W)$$ as

$$P(z,w)=\frac{1}{\lambda^w \mu^{1-w}}\exp\left[-\left(\frac{1}{\lambda}+\frac{1}{\mu}\right)z\right]$$

The above formulation is not completely rigorous as it is not a probability (I am searching for a better notation avoiding the dirac delta).

The likelihood function given the sample $$(z_1,w_1),\ldots,(z_n,w_n)$$ is then

$$L(\lambda,\mu)=\frac{1}{\lambda^{\sum_{i=1}^n w_i}\mu^{n-\sum_{i=1}^n w_i}}\exp\left[-\left(\frac{1}{\lambda}+\frac{1}{\mu}\right)\sum_{i=1}^n z_i\right]$$

Log-likelihood is

$$\ell(\lambda,\mu)=-\sum_{i=1}^n w_i\ln\lambda-\left(n-\sum_{i=1}^n w_i\right)\ln\mu--\left(\frac{1}{\lambda}+\frac{1}{\mu}\right)\sum_{i=1}^n z_i$$

For $$0<\bar w<1$$, solving for the stationary points of $$\ell(\lambda,\mu)$$ yields $$\hat\lambda=\frac{\sum_{i=1}^n z_i}{\sum_{i=1}^n w_i}=\frac{\bar z}{\bar w}\qquad,\qquad \hat\mu=\frac{\sum_{i=1}^n z_i}{n-\sum_{i=1}^n w_i}=\frac{\bar z}{1-\bar w}$$

So assuming $$0<\bar w<1$$, the unique MLE of $$(\lambda,\mu)$$ is $$(\hat\lambda,\hat\mu)$$.

But when $$\bar w\in\{0,1\}$$, the MLE does not exist.

• I still have two questions. (Q1) How to solve for the stationary points of $\ell(\lambda,\mu)$? (Q2) Why do we need to discuss the value range of $\bar{w}$? (In addition) Shouldn't the $\mu$ in your Log-likelihood is $\ln\mu$? – Tim Xu Mar 21 at 12:08
• You are right about the log-likelihood. I will edit. – StubbornAtom Mar 21 at 12:18
• @TimXu I just differentiated $\ell$ once wrt $\lambda$ and $\mu$ and set them equal to zero (for multiple parameters, one should ideally check whether the Hessian matrix of the log-likelihood is negative definite at $(\hat\lambda,\hat\mu)$, provided the derivatives exist). And you can see from the expression for $\hat\lambda,\hat\mu$ that they are not defined at $\bar w=0,1$. Hence the range distinction. – StubbornAtom Mar 21 at 12:35

The $$W=0$$ and the $$W=1$$ case can be combined by writing $$\lambda^{1-W}\mu^W$$.

First differentiating w.r.t. $$z$$ and forming the product the likelihood can be written as $$\prod_i^n{\frac{1}{\lambda^{w_i}\mu^{1-w_i}}}e^{-(\frac{1}{\lambda}+\frac{1}{\mu})z_i}$$. Taking logs gives the log likelihood $$=-\sum_i^n(w_i\ln{\lambda}+(1-w_i)\ln{\mu}+(\frac{1}{\lambda}+\frac{1}{\mu})z_i)$$.

Maximising wrt $$\lambda$$ and $$\mu$$ gives $$\lambda=\frac{\bar{z}}{\bar{w}}$$ and $$\mu=\frac{\bar{z}}{(1-\bar{w})}$$

Note if $$\lambda=\mu$$ then $$\bar{w}\approx\frac{1}{2}$$ so the estimates for $$\lambda$$ and $$\mu$$ become equal.

• Denote the log likelihood by $\log L$. First differentiating w.r.t. $\lambda$ and $\mu$, gives $\hat{\lambda}=\frac{\bar{z}}{\bar{w}}$, $\hat{\mu}=\frac{\bar{z}}{1-\bar{w}}$. To make sure they are the maximizers of the log likelihood. I calculate their second-order differentials w.r.t. $\lambda$ and $\mu$, and get that $\frac{\partial^2 \log L}{\partial \lambda^2}=-\sum_{i=1}^n(-\frac{w_i}{\lambda^2}+2\frac{z_i}{\lambda^3})$ and $\frac{\partial^2 \log L}{\partial \mu^2}=-\sum_{i=1}^n(-\frac{1-w_i}{\mu^2}+2\frac{z_i}{\mu^3})$. How do you know these two second-order differentials are negative? – Tim Xu Mar 21 at 11:44
• Direct substitution. for instance $\frac{\partial^2 \log{L}}{\partial \lambda^2}=-\frac{\bar{w}^3}{\bar{z}^2}$ – user121049 Mar 21 at 14:09
• You missed an $n$. It is $\frac{\partial ^2 \log{L}}{\partial \lambda^2} |_{\lambda=\frac{\bar z}{\bar w}}= -\sum_{i=1}^n (-\frac{w_i}{\lambda^2}+2\frac{z_i}{\lambda^3}) |_{\lambda=\frac{\bar z}{\bar w}}= (\frac{\sum_{i=1}^n w_i}{\lambda^2}-2\frac{\sum_{i=1}^n z_i}{\lambda^3}) |_{\lambda=\frac{\bar z}{\bar w}}= (\frac{n \bar w}{\lambda^2}-2\frac{n \bar z}{\lambda^3}) |_{\lambda=\frac{\bar z}{\bar w}}= \frac{n \bar w^3}{\bar z^2}-2\frac{n \bar z \bar w^3}{\bar z^3}= \frac{n \bar w^3}{\bar z^2}-2\frac{n \bar w^3}{\bar z^2}=-\frac{n \bar w^3}{\bar z^2}$ – Tim Xu Mar 22 at 6:41