5
$\begingroup$

I am studying the convolution method for creating the density function of two independent random variables and I am struggling with understanding how the bounds for integrals are created.

There is one example of a problem where I don't get the right answer:

Let $X$ be an exponential random variable with parameter $\lambda$ and $Y$ be a uniform random variable on $[0,1]$ independent of $X$. Find the probability density function of $X + Y$:

so I have two marginal functions

$\mathbf{\\ f_X(x)=\begin{cases} \lambda e^{-\lambda x} & x\geq 0\\ 0 & \text{otherwise} \end{cases}} \ \hspace{20pt} \ \mathbf{f_Y(y)=\begin{cases} 1 & 0\leq x\leq 1\\ 0 & \text{otherwise} \end{cases}}$

I am looking for $Z=X+Y$. My understanding is that two cases should be considered. First for $0\leq z\leq 1$ and second for $z>1$

for $\mathbf{0\leq z\leq 1\\ (f_X*f_Y)(z)=\int_{0}^{z} f_Y(z-y) \ f_X(y) \ dy = \int_0^z \lambda e^{-\lambda y} dy = [-e^{-\lambda y}]_{0}^{z} = 1 - e^{-\lambda z}}$

It seems that I got it correct to this point (in line with the given answer).

for $\mathbf{z>1}\\ \mathbf{(f_X*f_Y)(z)= {\color{Red} \int_1^z f_Y(z-y) \ f_X(y) \ dy = \int_1^z \lambda e^{-\lambda y} \, dy = [-e^{-\lambda y}]_{1}^{z} = 1 - e^{-\lambda z}=e^{-\lambda}(1-e^{-z})}}$

The result I got for $z>1$ is incorrect. I guess it is because I apply incorrect bounds to the integral and that's because I don't quite get the whole concept. I am looking for help with this example and more for general hint how to construct those intervals.

The correct answer should be:

$\mathbf{f_{X+Y}(z)=\begin{cases} 1-e^{-\lambda z} & 0 \leq z \leq 1 & \\ e^{-\lambda z}(e^\lambda -1) & z \geq 1 & \\ 0 & \text{otherwise} & \end{cases}}$

$\endgroup$
  • $\begingroup$ $X$ is ... exponential ... with rate parameter $\lambda$ .... $\endgroup$ – BruceET Sep 17 '15 at 22:15
  • 2
    $\begingroup$ corrected, thanks $\endgroup$ – Michal Sep 17 '15 at 22:33
5
$\begingroup$

If $z > 1$, we also require that $0 \le z-y \le 1$, or equivalently, $$z \ge y \ge z-1.$$ Thus your lower limit of integration is not correct: clearly, for a convolution integral of a uniform distribution with width $1$, your interval of integration must also have a width of $1$, and your integral limits do not satisfy this basic property.


Note that you would not be led astray if you expressed the densities in terms of indicator functions: $$f_X(x) = \lambda e^{-\lambda x} \mathbb{1}(x \ge 0), \quad f_Y(y) = \mathbb{1}(0 \le y \le 1).$$ Then our convolution is $$\begin{align*} f_Z(z) &= \int_{x = -\infty}^\infty f_X(x) f_Y(z-x) \, dx \\ &= \int_{x=-\infty}^\infty \lambda e^{-\lambda x} \mathbb{1}(x \ge 0) \mathbb{1}(0 \le z-x \le 1) \, dx \\ &= \int_{x = 0}^\infty \lambda e^{-\lambda x} \mathbb{1}(0 \le z-x \le 1) \, dx \\ &= \int_{x=0}^\infty \lambda e^{-\lambda x} \mathbb{1}(z \ge x \ge z-1) \, dx \\ &= \mathbb{1}(0 \le z \le 1) \int_{x=0}^z \lambda e^{-\lambda x} \, dx + \mathbb{1}(z > 1) \int_{x=z-1}^z \lambda e^{-\lambda x} \, dx . \end{align*}$$

The key point here is that we have a density $f_Y(z-x)$ which is nonzero only when $z-x \in [0,1]$. This is equivalent to saying that $x \in [z-1, z]$. But $x$ must also be nonnegative, because otherwise $f_X(x)$ would be zero. So in order for both densities to be positive, we must require $x \in [0,z]$ if $z \le 1$, and $x \in [z-1, z]$ when $z > 1$. We have to take the lower endpoint to be whichever is the larger of $0$ and $z-1$. This is precisely what the last expression above does. Now it is straightforward to evaluate the integrals to obtain $$\begin{align*} f_Z(z) &= \mathbb{1}(0 \le z \le 1) (1 - e^{-\lambda z}) + \mathbb{1}(z > 1) (e^{-\lambda (z-1)} - e^{-\lambda z}) \\ &= \begin{cases} 0, & z < 0 \\ 1-e^{-\lambda z}, & 0 \le z \le 1 \\ e^{-\lambda z}(e^\lambda-1) & z > 1. \end{cases} \end{align*}$$

$\endgroup$
  • $\begingroup$ ok thanks for the hint, I solved the integral and got the correct answer, however still struggling what determines the limits... $\mathbf{z>1}\\ \mathbf{(f_X*f_Y)(z)= \int_{z-1}^{z} f_Y(z-y) \ f_X(y) \ dy = \int_{z-1}^{z} \lambda e^{-\lambda y} \, dy = [-e^{-\lambda y}]_{z-1}^{z} = - e^{-\lambda z} + e^{-\lambda z}e^{\lambda}=-e^{-\lambda z}(e^{\lambda} -1)}$ $\endgroup$ – Michal Sep 18 '15 at 1:15
  • $\begingroup$ Check last equality. $\endgroup$ – BruceET Sep 18 '15 at 1:43
2
$\begingroup$

$$ (f_X * f_Y)(x) = \int_{-\infty}^\infty f_X(y) f_Y(x-y)\,dy = \int_A f_X(y) f_Y(x-y)\,dy $$ where $A$ is the set of values of $y$ for which both $$y\ge0\tag1$$ and $$0\le x-y \le 1.\tag2$$ Rearranging $(2)$ we get $$ y\le x \le y+1 \tag 3 $$ and that is equivalent to $$ x-1\le y \le x \tag 4 $$ since the first inequality in $(4)$ is equivalent to the second inequality in $(3)$ and vice-versa.

But we also need $(1)$, so we have $$ \max\{0,x-1\}\le y \le x. $$

Therefore we have $$ (f_X * f_Y)(x) = \begin{cases} \displaystyle \int_{\max\{0,x-1\}}^x f_X(y) f_Y(x-y)\,dy & \text{if }x\ge0, \\[10pt] 0 & \text{otherwise}, \end{cases} $$ or, if you like $$ (f_X * f_Y)(x) = \begin{cases} \displaystyle \int_{x-1}^x f_X(y) f_Y(x-y)\,dy & \text{if }x\ge1, \\[10pt] \displaystyle\int_0^x f_X(y) f_Y(x-y)\,dy & \text{if }0 < x < 1, \\[10pt] 0 & \text{otherwise}, \end{cases} $$ Since $f_Y(x-y)=1$ whenever $y$ is within those bounds, these can be trivially simplified. And after that, the integrals can easily be computed in closed form.

$\endgroup$
  • $\begingroup$ is it so that we always set x as the upper limit and then adjust/look for the lower? $\endgroup$ – Michal Sep 18 '15 at 1:13
  • $\begingroup$ No --- it just worked out that way in this case. ${}\qquad{}$ $\endgroup$ – Michael Hardy Sep 18 '15 at 2:25
1
$\begingroup$

Comment: I believe the answer for the density function is correct. Following the brief simulation below with $\lambda = 1/2$, I plot this density function (green) against the histogram of simulated values. Also, you can use your density function to verify that $E(Z)$ and $V(Z)$ have the obvious values.

The empirical cumulative distribution function (ECDF) of the simulated sample is also shown along with the CDF (green) corresponding to this density function.

 m = 10^4;  lam = 1/2
 x = rexp(m, lam);  y = runif(m, 0, 1);  z = x + y
 mean(z);  sd(z)
 ## 2.505842  # E(Z) = 2 + 1/2 = 2.5
 ## 2.02484
 sqrt(4 + 1/12)
 ## 2.020726

Notes: (a) $F_Z(z) = 1 - e^{-\lambda z}(e^\lambda-1)/\lambda,$ for $z \ge 1.$ The CDF must be continuous at $z = 1.$ (b) The number $m = 10000$ of iterations is relatively small to keep the file size for the ECDF within reasonable bounds, so the simulated values of $E(Z)$ and $SD(Z)$ are barely accurate to two places.

enter image description here

$\endgroup$
1
$\begingroup$

I made a youtube video based on this question explaining the concept very thoroughly.

Solution video: https://www.youtube.com/watch?v=f63e7uJeCOw

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.