# Exponential Likelihood Function

Suppose $$X_1, ..., X_n \stackrel{iid}{\sim}$$ Exponential(rate = $$\lambda$$) independent of $$Y_1, ..., Y_n \stackrel{iid}{\sim}$$ Exponential$$(1)$$.

Define $$Z_i \equiv \min\{X_i, Y_i\}$$

I want to find the maximum likelihood estimator for $$\lambda$$ in the following scenario: I observe $$Z_1, ..., Z_n$$ and $$Y_1, ..., Y_n$$ but NOT any of the $$X_i$$.

First I need to determine the likelihood and then maximize it over $$\theta > 0$$, but I'm not really sure of the right approach. I calculate the joint cdf as follows:

$$P(Z_i \leq z, Y_i \leq y) = \begin{cases} P(Y_i \leq y), & y \leq z \\ P(Y_i \leq z, Y_i \leq X_i) + P(Y_i \leq y, X_i \leq z, X_i < Y_i), & y > z\end{cases} \\ = \begin{cases} 1- e^{-y}, & y \leq z \\ 1-e^{-z} + (e^{-z}-e^{-y})(1-e^{-\lambda z}), & y > z \end{cases}$$

This is because $$Z_i \leq Y_i$$ always. Would the likelihood function therefore be:

$$L(\lambda |Y_i, Z_i, i \in \{1,...n\}) = \prod_{\{i : Y_i = Z_i\}} (1-e^{-Y_i}) \prod_{\{i:Y_i > Z_i\}} \lambda e^{-Y_i}e^{-\lambda Z_i}$$

splitting into the "discrete" and "continuous" parts? Or am I getting this wrong? Or should I be doing something like here or here? I should note my scenario is different than theirs, as intuitively at least, observing the magnitude of the difference between the minimum and the maximum (in the cases where $$Z_i$$ and $$Y_i$$ differ) should give us more information about $$\lambda$$, right?

• @StubbornAtom I can't find a closed form solution to the optimization problem I've set out in doing the above. Is this the right idea or am I implicitly supposed to do the problem outlined in the links I gave? If it's the same as the others, why is it not important that we observe the magnitude of the difference when there is a difference? Commented Apr 18, 2020 at 18:07

If you observe both $$Z_i$$ and $$Y_i$$, then when they are equal, you know $$X_i > Y_i$$. When they are not, you know $$X_i = Z_i$$. Therefore, your likelihood function is \begin{align*}\mathcal L(\lambda \mid \boldsymbol z, \boldsymbol y) &= \prod_{i=1}^n \left(f_X(z_i) \mathbb 1 (z_i \ne y_i) + (1 - F_X(y_i)) \mathbb 1 (z_i = y_i) \right) \\ &= \prod_{i=1}^n \left(\lambda e^{-\lambda z_i} \mathbb 1 (z_i \ne y_i) + e^{-\lambda y_i} \mathbb 1 (z_i = y_i) \right) \\ &= \lambda^{\sum_{i=1}^n \mathbb 1(z_i \ne y_i)} \prod_{i=1}^n e^{-\lambda z_i} \\ &= \lambda^{\sum_{i=1}^n \mathbb 1(z_i \ne y_1)} e^{-\lambda n \bar z}. \end{align*} Notice here that the density and survival functions we choose are for $$X$$, not on $$Y$$ or $$Z$$! Then the log-likelihood is $$\ell (\lambda \mid \boldsymbol z, \boldsymbol y) = ( \log \lambda ) \sum_{i=1}^n \mathbb 1 (z_i \ne y_i) - \lambda n \bar z,$$ and we solve for the extremum as usual, giving $$\hat \lambda = \frac{\sum_{i=1}^n \mathbb 1(z_i \ne y_i)}{n \bar z},$$ where the numerator counts the number of paired observations that are not equal, and the denominator is the sample total of $$z$$.

Simulation of this is straightforward and I invite you to try it out to confirm the estimator works. Here is code in Mathematica to perform the estimation based on a sample of size $$n$$ and any $$\lambda = t$$:

F[n_, t_] := RandomVariate[TransformedDistribution[{Min[x, y], y},
{Distributed[x, ExponentialDistribution[t]],
Distributed[y, ExponentialDistribution[1]]}], n]

T[d_] := Length[Select[d, #[[1]] != #[[2]] &]]/Total[First /@ d]

T[F[10^6, Pi]]


The last expression evaluates $$\hat \lambda$$ for $$n = 10^6$$ and $$\lambda = \pi$$. I got $$3.14452$$ when I ran it.

• In the likelihood, why is there a $\lambda$ in the $y_i$ part? Should that not be equal to simply $y_i$? Commented Apr 18, 2020 at 20:24
• @qp212223 As I stated, I am looking at the density and survival of $X$, not $Y$ or $Z$. You are thinking in terms of the likelihood of the joint derived variables. I'm looking at the likelihood on the information we can extract about the original variable $X$ through $Y$ and $Z$. Commented Apr 18, 2020 at 20:26
• Your first expression suggests that conditioned on $z_i \not= y_i$ you have $Z_i =X_i \sim \text{ Exp}(\lambda)$. If you simulate this (discarding cases where $z_i=y_i$) then I think you will find the conditional distribution of $Z_i=X_i$ will be $\text{ Exp}(\lambda+1)$ Commented Apr 18, 2020 at 22:42
• With my correction to my answer, I now get the same result as yours. It still think I am correct about the conditional density, but it makes no difference to the maximum likelihood estimator because it simply introduces a multiplicative term $e^{-\sum z_i}$ to the likelihood which does not depend on $\lambda$ Commented Apr 19, 2020 at 0:17

I would guess that the useful information is in the values of $$Z_i$$ and how often $$Y_i=Z_i$$ or not (perhaps call this $$Q$$); the actual values of $$Y_i$$ may not help beyond this.

I think you could show $$Z_1, ..., Z_n \stackrel{iid}{\sim} \text{ Exponential(rate }= \lambda+1)$$ and independently $$Q \sim \text{ Binomial}\left(n,\frac{1}{\lambda+1}\right)$$. In that case the useful likelihood of observing $$z_1,\ldots,z_n$$ and $$q$$ (so ignoring parts related to $$Y_i-Z_i$$ when that is positive) would be proportional to

$$(\lambda+1)^ne^{-\sum(\lambda+1) z_i} {n \choose q}\frac{\lambda^{n-q}}{(\lambda+1)^n}={n \choose q} \lambda^{n-q} e^{-(\lambda+1)\sum z_i}$$

with logarithm a constant plus $$(n-q) \log(\lambda) -(\lambda+1)\sum z_i$$

and derivative of the logarithm with respect to $$\lambda$$ $$\frac{n-q}{\lambda} - \sum z_i$$

and the maximum likelihood estimator $$\hat \lambda = \frac{n-q}{\sum z_i}$$

• How do you justify that $Q$ is independent of the $Z_i$? Commented Apr 18, 2020 at 20:35
• @angryavian - through the memoryless property of exponential distributions and Poisson processes; if you know that both $X_i$ and $Y_i$ are greater than a particular value $k$ then the conditional probability $Y_i < X_i$ is still $\frac1{\lambda+1}$ no matter what the value of $k$ Commented Apr 18, 2020 at 22:15
• @Henry Have you tried simulating your MLE? I could not get a reasonable estimate with your result; the denominator is too large. It is also obvious that since $q \ge 0$ and $z_i > 0$, your estimator is bounded above by $1$. But no such restriction on $\lambda$ is stipulated. Commented Apr 18, 2020 at 22:57
• @heropup - I see your point about the bound of $1$ and will investigate further Commented Apr 18, 2020 at 23:31
• @heropup - it seems I made an error in the right-hand side of the first expression, with consequences for the MLE, and I now have the same answer as you, despite the different starting likelihood - thank you for your comments Commented Apr 19, 2020 at 0:11

Would this be $$\prod_{\{i: Y_i = Z_i\}} \frac{1}{\lambda +1} \prod_{\{i: Y_i > Z_i\}} e^{-Y_i}\lambda e^{-\lambda Z_i}$$

where we just have the point mass/probability of equality contributing when $$Y_i = Z_i$$ and the joint density contributing otherwise. Can someone please provide some insight?

• I think this may be $\prod\limits_{\{i: Y_i = Z_i\}} \left(\frac{1}{\lambda +1} (\lambda+1)e^{-(\lambda+1)Z_i} \right)\prod\limits_{\{i: Y_i > Z_i\}} \left( \frac{\lambda}{\lambda +1} e^{-(Y_i-Z_i)} (\lambda+1)e^{-(\lambda+1)Z_i} \right)$ Commented Apr 18, 2020 at 20:15