Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

definition: Laplace distribution $Lap(\mu, b)$ with mean $\mu$ and a scaling paramter $b$ is defined as $$f_X(x;\mu, b) = \frac{1}{2b} \exp \left( -\frac{|x-\mu|}{b} \right)$$

The standard Laplace distribution is a simplifed version where $\mu = 0$ and $b=1$.

relation with exponential distribution: As shown at Laplace distribution article on Wikipedia, the Laplace random variable $Z\sim Lap(0, 1/\lambda)$ is obtained by the difference $Z = X-Y$ of two iid exponential random variable $X, Y\sim Exp(\lambda)$.

general case: As pointed out in Proposition 2.2.3 from The Laplace distribution and generalizations: a revisit with applications to Communications, Economics, Engineering and Finance by Kotz et al, a Laplace random variable $Z$ is obtained by $$Z=\sum w_i X_i$$ where each $w_i$ takes on values $\pm 1$ with probabilities 1/2 and $X_i$ is standard exponential random variables.

question: I'm interested in the generalization of this representation so that $$Z = \sum w_i X_i$$ where each $w_i$ is any real number and each $X_i$ is independently sampled from $Exp(1)$. If $w_i >0$, then it defines the hypoexponential distribution since the the summation becomes $Z=\sum Exp(1/w_i)$ due to the closure under scaling. Is there any probability distribution about a weighted sum where each $w_i$ is any real number?

share|cite|improve this question
So what is your actual question? – Dilip Sarwate Nov 25 '11 at 23:44
up vote 1 down vote accepted

a Laplace random variable $Z$ is obtained by $$Z=\sum w_iX_i$$ where each $w_i$ takes on values $\pm 1$ with probabilities $1/2$ and $X_i$ is standard exponential random variables.

Is this statement really correct? I can see why it holds when $Z = w_1X_1$ because the conditional density of $Z$ given $w_1 = +1$ is $\exp(-x)\mathbf 1_{[0,\infty)}$ while the conditional density of $Z$ given $w_1 = -1$ is $\exp(x)\mathbf 1_{(-\infty,0]}$, and so by the law of total probability, we have $$f_Z(x) = \frac{1}{2}\exp(-x)\mathbf 1_{[0,\infty)} + \frac{1}{2}\exp(x)\mathbf 1_{(-\infty,0]} = \frac{1}{2}\exp(-|x|), -\infty < x < \infty,$$ which, ignoring the value at $x=0$, is a Laplacian density. But by the same argument, the conditional density of $w_1X_1 + w_2X_2$ is a Gamma density of order $2$ when $w_1 = w_2 = +1$, and this should show up in the unconditional pdf as well.

Turning to the OP's question about the density of $\sum w_iX_i$ where the $w_i$ are arbitrary real numbers and the $X_i$ are independent exponential random variables with mean $1$, note that the moment-generating function of $w_iX_i$ is $E[\exp(tw_iX_i)] = (1-w_it)^{-1}$, we have $$E[\exp(tZ)] = \prod_{i=1}^n \frac{1}{1 - w_it}.$$ Assuming that the $|w_i|$ all are distinct real numbers, this can be expanded via partial fractions into a weighted sum of terms of the form $(1-w_it)^{-1}$, and so the density of $Z$ is a weighted sum of exponential densities $w_i^{-1}\exp(-x/w_i)\mathbf 1_{[0,\infty)}$ (when $w_i > 0$) and densities $|w_i|^{-1}\exp(-x/w_i)\mathbf 1_{(-\infty,0]}$ (when $w_i < 0$).

share|cite|improve this answer
Just to make it sure: When you said '$Z$ is a weighted sum of exponential densities,' I thought you meant the weight can be obtained from something similar to that of article but you did not mean that the weight of each term is $|w_i|$, right? – Federico Magallanez Nov 27 '11 at 22:37
@FedericoMagallanez The weights in the "weighted sum of densities" are whatever are given by the partial fraction expansion of the product, and not the $w_i$ that are the weights on the $X_i$. For example, $$\frac{1}{(1-w_1t)(1-w_2t)} = \frac{w_1/(w_1-w2)}{1-w_1t}-\frac{w_2/(w_1-w2)}{1-w_2t}$$ – Dilip Sarwate Nov 27 '11 at 23:14

In other words, gives two independent random variables $X$ and $Y$, distributed according to hypoexponential distribution with parameters $\{ w_1, w_2, \ldots, w_n \}$ and $\{v_1,v_2, \ldots, v_m \}$ respectively, you are asking to determine the distribution of $Z=X-Y$.

Let $\Theta_X$, and $\Theta_Y$ denote matrices from the probability density functions of respective hypoexponential distributions, see wiki page: $$ f_X(x) = - \langle\vec{\alpha}_n, \exp(x \Theta_X) \Theta_X, \vec{1}_n \rangle \cdot [ x > 0 ] \qquad \qquad f_Y(y) = - \langle\vec{\alpha}_m, \exp(y \Theta_Y) \Theta_Y, \vec{1}_m \rangle \cdot [ y > 0 ] $$ where $(\alpha_n)_i = \delta_{i,1}$, $ (\vec{1}_n)_i = 1$ and $(\alpha_m)_j = \delta_{j,1}$, $ (\vec{1}_m)_j = 1$, $i=1,\ldots,n$, and $j=1,\ldots,m$.

Then $$ \begin{eqnarray} f_Z(z) &=& \int_{-\infty}^\infty f_X(z+y) f_Y(y) \mathrm{d} y = \int_{\max(-z,0)}^\infty f_X(z+y) f_Y(y) \mathrm{d} y \\ &=& \int_{\max(-z,0)}^\infty \langle \vec{\alpha}_n \otimes \vec{\alpha}_m, \left( \mathrm{e}^{(z+y) \Theta_X} \Theta_X \right) \otimes \left( \mathrm{e}^{y \Theta_Y} \Theta_Y\right), \vec{\mathbf{1}}_n \otimes \vec{\mathbf{1}}_m \rangle \mathrm{d} y \\ &=& \int_{0}^\infty \langle \vec{\alpha}_n \otimes \vec{\alpha}_m, \left( \mathrm{e}^{(\max(z,0)+y) \Theta_X} \Theta_X \right) \otimes \left( \mathrm{e}^{(\max(-z,0) + y) \Theta_Y} \Theta_Y\right), \vec{\mathbf{1}}_n \otimes \vec{\mathbf{1}}_m \rangle \mathrm{d} y \\ &=& \langle \vec{\alpha}_n \otimes \vec{\alpha}_m, \left( \mathrm{e}^{\max(z,0) \Theta_X} \otimes \mathrm{e}^{(\max(-z,0) ) \Theta_Y} \right) \cdot \left( \int_{0}^\infty \mathrm{e}^{y \Theta_X} \Theta_X \otimes \mathrm{e}^{y \Theta_Y} \Theta_Y \mathrm{d} y \right), \vec{\mathbf{1}}_n \otimes \vec{\mathbf{1}}_m \rangle \end{eqnarray} $$ The formula above tells the density function for $Z$ will be piecewise, much like Laplace distribution, with functional form of $X$ variate for $z>0$ and functional form of $Y$ variate for $z<0$.

Example: Consider an example with $n=2$ and $m=2$, and $\{w_1,w_2\} = \{1,2\}$, and $\{v_1,v_2\} = \{1,1\}$. Corresponding matrices are $$ \Theta_X = \left( \begin{array}{cc} -1 & 1 \\ 0 & -2 \end{array} \right) \qquad \Theta_Y = \left( \begin{array}{cc} -1 & 1 \\ 0 & -1 \end{array} \right) $$ Then $$ \exp\left(x \Theta_X \right) = \left( \begin{array}{cc} e^{-x} & e^{-x}-e^{-2 x} \\ 0 & e^{-2 x} \\ \end{array} \right) \qquad \exp\left(y \Theta_Y \right) = \left( \begin{array}{cc} e^{-y} & e^{-y} y \\ 0 & e^{-y} \\ \end{array} \right) $$ $$ \exp\left(x \Theta_X \right) \Theta_X = \left( \begin{array}{cc} -e^{-x} & 2 e^{-2 x}-e^{-x} \\ 0 & -2 e^{-2 x} \\ \end{array} \right) \qquad \exp\left(y \Theta_Y \right) \Theta_Y = \left( \begin{array}{cc} -e^{-y} & e^{-y}-e^{-y} y \\ 0 & -e^{-y} \\ \end{array} \right) $$ Using Kronecker product, $$ \int_{0}^\infty \mathrm{e}^{y \Theta_X} \Theta_X \otimes \mathrm{e}^{y \Theta_Y} \Theta_Y \mathrm{d} y = \left( \begin{array}{cccc} \frac{1}{2} & -\frac{1}{4} & -\frac{1}{6} & \frac{7}{36} \\ 0 & \frac{1}{2} & 0 & -\frac{1}{6} \\ 0 & 0 & \frac{2}{3} & -\frac{4}{9} \\ 0 & 0 & 0 & \frac{2}{3} \\ \end{array} \right) $$ Combining things, with little algebra we get: $$ f_Z(z) = \left\{ \begin{array}{cc} \frac{5}{18} & z=0 \\ \frac{1}{18} \mathrm{e}^{-z} \left(9-4 \mathrm{e}^{-z}\right) & z>0 \\ \frac{1}{18} \mathrm{e}^z (5-6 z) & z < 0 \\ \end{array} \right. $$

share|cite|improve this answer
Thank you! As I don't have enoug background on this topic, let me ask you a few things. I assume $\langle a,b\rangle$ is the inner product of a and b. But your bracket notation has three arguments. Does it mean anything other than inner product? – Federico Magallanez Nov 26 '11 at 8:04
Yes, $\langle a, T, b \rangle$ denotes inner product $\sum_{i,j} a_i T_{ij} b_j$. $a \otimes b$ denotes a tensor such that $(a \otimes b)_{ij} = a_i b_j$. Also $\langle a \otimes b, T \otimes S, c \otimes d \rangle = \sum_{i,j,m,n} a_i b_m T_{ij} S_{mn} c_j d_n$ which is just a product of inner products. – Sasha Nov 26 '11 at 14:32
(1) Can you explain how Kronecker product kicks in while you substitute $f_X(z+y)$ and $f_Y(y)$ in the integral? (2) Can we have a closed form of the final answer if we assume that all the $w_1$, ..., $w_n$ and $v_1$, ..., $v_m$ are distinct? – Federico Magallanez Nov 26 '11 at 15:33
@user18526 (1) The Kronecker product kicks is because $\langle a, T, b \rangle \langle c, S, d \rangle = \langle a \otimes c, T \otimes S, b \otimes d$. For the (2) it is definitely possible, and the simplest way is to use the moment generating function technique. $\mathcal{M}_Z(t) = \mathcal{M}_X(t) \mathcal{M}_Y(-t)$. Since $\mathcal{M}_Z(t)$ is a rational function of $t$ as a product of rational functions, do partial fraction decomposition, and read off linear combinations of exponentials and reflected exponentials. – Sasha Nov 26 '11 at 18:26
Thank you for your answer. But as a layman, I'm afraid I become more puzzled by your answer. Is there any textbook or reference on why the probability density of a hypoexponential with parameter $\Theta$ can be written as $f(x)=-\boldsymbol{\alpha}e^{x\Theta}\Theta\boldsymbol{1}$ in the wikipedia article on hypoexponential distribution? – Federico Magallanez Nov 26 '11 at 21:07

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.