Suppose $X_1, ..., X_n \stackrel{iid}{\sim}$ Exponential(rate = $\lambda$) independent of $Y_1, ..., Y_n \stackrel{iid}{\sim}$ Exponential$(1)$.

Define $Z_i \equiv \min\{X_i, Y_i\}$

I want to find the maximum likelihood estimator for $\lambda$ in the following scenario: I observe $Z_1, ..., Z_n$ and $Y_1, ..., Y_n$ but NOT any of the $X_i$.

First I need to determine the likelihood and then maximize it over $\theta > 0$, but I'm not really sure of the right approach. I calculate the joint cdf as follows:

$$P(Z_i \leq z, Y_i \leq y) = \begin{cases} P(Y_i \leq y), & y \leq z \\ P(Y_i \leq z, Y_i \leq X_i) + P(Y_i \leq y, X_i \leq z, X_i < Y_i), & y > z\end{cases} \\ = \begin{cases} 1- e^{-y}, & y \leq z \\ 1-e^{-z} + (e^{-z}-e^{-y})(1-e^{-\lambda z}), & y > z \end{cases}$$

This is because $Z_i \leq Y_i$ always. Would the likelihood function therefore be:

$$L(\lambda |Y_i, Z_i, i \in \{1,...n\}) = \prod_{\{i : Y_i = Z_i\}} (1-e^{-Y_i}) \prod_{\{i:Y_i > Z_i\}} \lambda e^{-Y_i}e^{-\lambda Z_i}$$

splitting into the "discrete" and "continuous" parts? Or am I getting this wrong? Or should I be doing something like here or here? I should note my scenario is different than theirs, as intuitively at least, observing the magnitude of the difference between the minimum and the maximum (in the cases where $Z_i$ and $Y_i$ differ) should give us more information about $\lambda$, right?

  • $\begingroup$ @StubbornAtom I can't find a closed form solution to the optimization problem I've set out in doing the above. Is this the right idea or am I implicitly supposed to do the problem outlined in the links I gave? If it's the same as the others, why is it not important that we observe the magnitude of the difference when there is a difference? $\endgroup$
    – qp212223
    Commented Apr 18, 2020 at 18:07

3 Answers 3


If you observe both $Z_i$ and $Y_i$, then when they are equal, you know $X_i > Y_i$. When they are not, you know $X_i = Z_i$. Therefore, your likelihood function is $$\begin{align*}\mathcal L(\lambda \mid \boldsymbol z, \boldsymbol y) &= \prod_{i=1}^n \left(f_X(z_i) \mathbb 1 (z_i \ne y_i) + (1 - F_X(y_i)) \mathbb 1 (z_i = y_i) \right) \\ &= \prod_{i=1}^n \left(\lambda e^{-\lambda z_i} \mathbb 1 (z_i \ne y_i) + e^{-\lambda y_i} \mathbb 1 (z_i = y_i) \right) \\ &= \lambda^{\sum_{i=1}^n \mathbb 1(z_i \ne y_i)} \prod_{i=1}^n e^{-\lambda z_i} \\ &= \lambda^{\sum_{i=1}^n \mathbb 1(z_i \ne y_1)} e^{-\lambda n \bar z}. \end{align*}$$ Notice here that the density and survival functions we choose are for $X$, not on $Y$ or $Z$! Then the log-likelihood is $$\ell (\lambda \mid \boldsymbol z, \boldsymbol y) = ( \log \lambda ) \sum_{i=1}^n \mathbb 1 (z_i \ne y_i) - \lambda n \bar z,$$ and we solve for the extremum as usual, giving $$\hat \lambda = \frac{\sum_{i=1}^n \mathbb 1(z_i \ne y_i)}{n \bar z},$$ where the numerator counts the number of paired observations that are not equal, and the denominator is the sample total of $z$.

Simulation of this is straightforward and I invite you to try it out to confirm the estimator works. Here is code in Mathematica to perform the estimation based on a sample of size $n$ and any $\lambda = t$:

F[n_, t_] := RandomVariate[TransformedDistribution[{Min[x, y], y},
             {Distributed[x, ExponentialDistribution[t]], 
             Distributed[y, ExponentialDistribution[1]]}], n]

T[d_] := Length[Select[d, #[[1]] != #[[2]] &]]/Total[First /@ d]

T[F[10^6, Pi]]

The last expression evaluates $\hat \lambda$ for $n = 10^6$ and $\lambda = \pi$. I got $3.14452$ when I ran it.

  • $\begingroup$ In the likelihood, why is there a $\lambda$ in the $y_i$ part? Should that not be equal to simply $y_i$? $\endgroup$
    – qp212223
    Commented Apr 18, 2020 at 20:24
  • $\begingroup$ @qp212223 As I stated, I am looking at the density and survival of $X$, not $Y$ or $Z$. You are thinking in terms of the likelihood of the joint derived variables. I'm looking at the likelihood on the information we can extract about the original variable $X$ through $Y$ and $Z$. $\endgroup$
    – heropup
    Commented Apr 18, 2020 at 20:26
  • $\begingroup$ Your first expression suggests that conditioned on $z_i \not= y_i$ you have $Z_i =X_i \sim \text{ Exp}(\lambda)$. If you simulate this (discarding cases where $z_i=y_i$) then I think you will find the conditional distribution of $Z_i=X_i$ will be $\text{ Exp}(\lambda+1)$ $\endgroup$
    – Henry
    Commented Apr 18, 2020 at 22:42
  • $\begingroup$ With my correction to my answer, I now get the same result as yours. It still think I am correct about the conditional density, but it makes no difference to the maximum likelihood estimator because it simply introduces a multiplicative term $e^{-\sum z_i}$ to the likelihood which does not depend on $\lambda$ $\endgroup$
    – Henry
    Commented Apr 19, 2020 at 0:17

I would guess that the useful information is in the values of $Z_i$ and how often $Y_i=Z_i$ or not (perhaps call this $Q$); the actual values of $Y_i$ may not help beyond this.

I think you could show $Z_1, ..., Z_n \stackrel{iid}{\sim} \text{ Exponential(rate }= \lambda+1)$ and independently $Q \sim \text{ Binomial}\left(n,\frac{1}{\lambda+1}\right)$. In that case the useful likelihood of observing $z_1,\ldots,z_n$ and $q$ (so ignoring parts related to $Y_i-Z_i$ when that is positive) would be proportional to

$$(\lambda+1)^ne^{-\sum(\lambda+1) z_i} {n \choose q}\frac{\lambda^{n-q}}{(\lambda+1)^n}={n \choose q} \lambda^{n-q} e^{-(\lambda+1)\sum z_i}$$

with logarithm a constant plus $$(n-q) \log(\lambda) -(\lambda+1)\sum z_i$$

and derivative of the logarithm with respect to $\lambda$ $$\frac{n-q}{\lambda} - \sum z_i$$

and the maximum likelihood estimator $$\hat \lambda = \frac{n-q}{\sum z_i}$$

  • $\begingroup$ How do you justify that $Q$ is independent of the $Z_i$? $\endgroup$
    – angryavian
    Commented Apr 18, 2020 at 20:35
  • $\begingroup$ @angryavian - through the memoryless property of exponential distributions and Poisson processes; if you know that both $X_i$ and $Y_i$ are greater than a particular value $k$ then the conditional probability $Y_i < X_i$ is still $\frac1{\lambda+1}$ no matter what the value of $k$ $\endgroup$
    – Henry
    Commented Apr 18, 2020 at 22:15
  • $\begingroup$ @Henry Have you tried simulating your MLE? I could not get a reasonable estimate with your result; the denominator is too large. It is also obvious that since $q \ge 0$ and $z_i > 0$, your estimator is bounded above by $1$. But no such restriction on $\lambda$ is stipulated. $\endgroup$
    – heropup
    Commented Apr 18, 2020 at 22:57
  • $\begingroup$ @heropup - I see your point about the bound of $1$ and will investigate further $\endgroup$
    – Henry
    Commented Apr 18, 2020 at 23:31
  • $\begingroup$ @heropup - it seems I made an error in the right-hand side of the first expression, with consequences for the MLE, and I now have the same answer as you, despite the different starting likelihood - thank you for your comments $\endgroup$
    – Henry
    Commented Apr 19, 2020 at 0:11

Would this be $$\prod_{\{i: Y_i = Z_i\}} \frac{1}{\lambda +1} \prod_{\{i: Y_i > Z_i\}} e^{-Y_i}\lambda e^{-\lambda Z_i} $$

where we just have the point mass/probability of equality contributing when $Y_i = Z_i$ and the joint density contributing otherwise. Can someone please provide some insight?

  • $\begingroup$ I think this may be $\prod\limits_{\{i: Y_i = Z_i\}} \left(\frac{1}{\lambda +1} (\lambda+1)e^{-(\lambda+1)Z_i} \right)\prod\limits_{\{i: Y_i > Z_i\}} \left( \frac{\lambda}{\lambda +1} e^{-(Y_i-Z_i)} (\lambda+1)e^{-(\lambda+1)Z_i} \right)$ $\endgroup$
    – Henry
    Commented Apr 18, 2020 at 20:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .