# Generalized Laplace distribution?

definition: Laplace distribution $Lap(\mu, b)$ with mean $\mu$ and a scaling paramter $b$ is defined as $$f_X(x;\mu, b) = \frac{1}{2b} \exp \left( -\frac{|x-\mu|}{b} \right)$$

The standard Laplace distribution is a simplifed version where $\mu = 0$ and $b=1$.

relation with exponential distribution: As shown at Laplace distribution article on Wikipedia, the Laplace random variable $Z\sim Lap(0, 1/\lambda)$ is obtained by the difference $Z = X-Y$ of two iid exponential random variable $X, Y\sim Exp(\lambda)$.

general case: As pointed out in Proposition 2.2.3 from The Laplace distribution and generalizations: a revisit with applications to Communications, Economics, Engineering and Finance by Kotz et al, a Laplace random variable $Z$ is obtained by $$Z=\sum w_i X_i$$ where each $w_i$ takes on values $\pm 1$ with probabilities 1/2 and $X_i$ is standard exponential random variables.

question: I'm interested in the generalization of this representation so that $$Z = \sum w_i X_i$$ where each $w_i$ is any real number and each $X_i$ is independently sampled from $Exp(1)$. If $w_i >0$, then it defines the hypoexponential distribution since the the summation becomes $Z=\sum Exp(1/w_i)$ due to the closure under scaling. Is there any probability distribution about a weighted sum where each $w_i$ is any real number?

-
So what is your actual question? – Dilip Sarwate Nov 25 '11 at 23:44

a Laplace random variable $Z$ is obtained by $$Z=\sum w_iX_i$$ where each $w_i$ takes on values $\pm 1$ with probabilities $1/2$ and $X_i$ is standard exponential random variables.

Is this statement really correct? I can see why it holds when $Z = w_1X_1$ because the conditional density of $Z$ given $w_1 = +1$ is $\exp(-x)\mathbf 1_{[0,\infty)}$ while the conditional density of $Z$ given $w_1 = -1$ is $\exp(x)\mathbf 1_{(-\infty,0]}$, and so by the law of total probability, we have $$f_Z(x) = \frac{1}{2}\exp(-x)\mathbf 1_{[0,\infty)} + \frac{1}{2}\exp(x)\mathbf 1_{(-\infty,0]} = \frac{1}{2}\exp(-|x|), -\infty < x < \infty,$$ which, ignoring the value at $x=0$, is a Laplacian density. But by the same argument, the conditional density of $w_1X_1 + w_2X_2$ is a Gamma density of order $2$ when $w_1 = w_2 = +1$, and this should show up in the unconditional pdf as well.

Turning to the OP's question about the density of $\sum w_iX_i$ where the $w_i$ are arbitrary real numbers and the $X_i$ are independent exponential random variables with mean $1$, note that the moment-generating function of $w_iX_i$ is $E[\exp(tw_iX_i)] = (1-w_it)^{-1}$, we have $$E[\exp(tZ)] = \prod_{i=1}^n \frac{1}{1 - w_it}.$$ Assuming that the $|w_i|$ all are distinct real numbers, this can be expanded via partial fractions into a weighted sum of terms of the form $(1-w_it)^{-1}$, and so the density of $Z$ is a weighted sum of exponential densities $w_i^{-1}\exp(-x/w_i)\mathbf 1_{[0,\infty)}$ (when $w_i > 0$) and densities $|w_i|^{-1}\exp(-x/w_i)\mathbf 1_{(-\infty,0]}$ (when $w_i < 0$).

-
Just to make it sure: When you said '$Z$ is a weighted sum of exponential densities,' I thought you meant the weight can be obtained from something similar to that of article but you did not mean that the weight of each term is $|w_i|$, right? – Federico Magallanez Nov 27 '11 at 22:37
@FedericoMagallanez The weights in the "weighted sum of densities" are whatever are given by the partial fraction expansion of the product, and not the $w_i$ that are the weights on the $X_i$. For example, $$\frac{1}{(1-w_1t)(1-w_2t)} = \frac{w_1/(w_1-w2)}{1-w_1t}-\frac{w_2/(w_1-w2)}{1-w_2t}$$ – Dilip Sarwate Nov 27 '11 at 23:14

In other words, gives two independent random variables $X$ and $Y$, distributed according to hypoexponential distribution with parameters $\{ w_1, w_2, \ldots, w_n \}$ and $\{v_1,v_2, \ldots, v_m \}$ respectively, you are asking to determine the distribution of $Z=X-Y$.

Let $\Theta_X$, and $\Theta_Y$ denote matrices from the probability density functions of respective hypoexponential distributions, see wiki page: $$f_X(x) = - \langle\vec{\alpha}_n, \exp(x \Theta_X) \Theta_X, \vec{1}_n \rangle \cdot [ x > 0 ] \qquad \qquad f_Y(y) = - \langle\vec{\alpha}_m, \exp(y \Theta_Y) \Theta_Y, \vec{1}_m \rangle \cdot [ y > 0 ]$$ where $(\alpha_n)_i = \delta_{i,1}$, $(\vec{1}_n)_i = 1$ and $(\alpha_m)_j = \delta_{j,1}$, $(\vec{1}_m)_j = 1$, $i=1,\ldots,n$, and $j=1,\ldots,m$.

Then $$\begin{eqnarray} f_Z(z) &=& \int_{-\infty}^\infty f_X(z+y) f_Y(y) \mathrm{d} y = \int_{\max(-z,0)}^\infty f_X(z+y) f_Y(y) \mathrm{d} y \\ &=& \int_{\max(-z,0)}^\infty \langle \vec{\alpha}_n \otimes \vec{\alpha}_m, \left( \mathrm{e}^{(z+y) \Theta_X} \Theta_X \right) \otimes \left( \mathrm{e}^{y \Theta_Y} \Theta_Y\right), \vec{\mathbf{1}}_n \otimes \vec{\mathbf{1}}_m \rangle \mathrm{d} y \\ &=& \int_{0}^\infty \langle \vec{\alpha}_n \otimes \vec{\alpha}_m, \left( \mathrm{e}^{(\max(z,0)+y) \Theta_X} \Theta_X \right) \otimes \left( \mathrm{e}^{(\max(-z,0) + y) \Theta_Y} \Theta_Y\right), \vec{\mathbf{1}}_n \otimes \vec{\mathbf{1}}_m \rangle \mathrm{d} y \\ &=& \langle \vec{\alpha}_n \otimes \vec{\alpha}_m, \left( \mathrm{e}^{\max(z,0) \Theta_X} \otimes \mathrm{e}^{(\max(-z,0) ) \Theta_Y} \right) \cdot \left( \int_{0}^\infty \mathrm{e}^{y \Theta_X} \Theta_X \otimes \mathrm{e}^{y \Theta_Y} \Theta_Y \mathrm{d} y \right), \vec{\mathbf{1}}_n \otimes \vec{\mathbf{1}}_m \rangle \end{eqnarray}$$ The formula above tells the density function for $Z$ will be piecewise, much like Laplace distribution, with functional form of $X$ variate for $z>0$ and functional form of $Y$ variate for $z<0$.

Example: Consider an example with $n=2$ and $m=2$, and $\{w_1,w_2\} = \{1,2\}$, and $\{v_1,v_2\} = \{1,1\}$. Corresponding matrices are $$\Theta_X = \left( \begin{array}{cc} -1 & 1 \\ 0 & -2 \end{array} \right) \qquad \Theta_Y = \left( \begin{array}{cc} -1 & 1 \\ 0 & -1 \end{array} \right)$$ Then $$\exp\left(x \Theta_X \right) = \left( \begin{array}{cc} e^{-x} & e^{-x}-e^{-2 x} \\ 0 & e^{-2 x} \\ \end{array} \right) \qquad \exp\left(y \Theta_Y \right) = \left( \begin{array}{cc} e^{-y} & e^{-y} y \\ 0 & e^{-y} \\ \end{array} \right)$$ $$\exp\left(x \Theta_X \right) \Theta_X = \left( \begin{array}{cc} -e^{-x} & 2 e^{-2 x}-e^{-x} \\ 0 & -2 e^{-2 x} \\ \end{array} \right) \qquad \exp\left(y \Theta_Y \right) \Theta_Y = \left( \begin{array}{cc} -e^{-y} & e^{-y}-e^{-y} y \\ 0 & -e^{-y} \\ \end{array} \right)$$ Using Kronecker product, $$\int_{0}^\infty \mathrm{e}^{y \Theta_X} \Theta_X \otimes \mathrm{e}^{y \Theta_Y} \Theta_Y \mathrm{d} y = \left( \begin{array}{cccc} \frac{1}{2} & -\frac{1}{4} & -\frac{1}{6} & \frac{7}{36} \\ 0 & \frac{1}{2} & 0 & -\frac{1}{6} \\ 0 & 0 & \frac{2}{3} & -\frac{4}{9} \\ 0 & 0 & 0 & \frac{2}{3} \\ \end{array} \right)$$ Combining things, with little algebra we get: $$f_Z(z) = \left\{ \begin{array}{cc} \frac{5}{18} & z=0 \\ \frac{1}{18} \mathrm{e}^{-z} \left(9-4 \mathrm{e}^{-z}\right) & z>0 \\ \frac{1}{18} \mathrm{e}^z (5-6 z) & z < 0 \\ \end{array} \right.$$

-
Thank you! As I don't have enoug background on this topic, let me ask you a few things. I assume $\langle a,b\rangle$ is the inner product of a and b. But your bracket notation has three arguments. Does it mean anything other than inner product? – Federico Magallanez Nov 26 '11 at 8:04
Yes, $\langle a, T, b \rangle$ denotes inner product $\sum_{i,j} a_i T_{ij} b_j$. $a \otimes b$ denotes a tensor such that $(a \otimes b)_{ij} = a_i b_j$. Also $\langle a \otimes b, T \otimes S, c \otimes d \rangle = \sum_{i,j,m,n} a_i b_m T_{ij} S_{mn} c_j d_n$ which is just a product of inner products. – Sasha Nov 26 '11 at 14:32
(1) Can you explain how Kronecker product kicks in while you substitute $f_X(z+y)$ and $f_Y(y)$ in the integral? (2) Can we have a closed form of the final answer if we assume that all the $w_1$, ..., $w_n$ and $v_1$, ..., $v_m$ are distinct? – Federico Magallanez Nov 26 '11 at 15:33
@user18526 (1) The Kronecker product kicks is because $\langle a, T, b \rangle \langle c, S, d \rangle = \langle a \otimes c, T \otimes S, b \otimes d$. For the (2) it is definitely possible, and the simplest way is to use the moment generating function technique. $\mathcal{M}_Z(t) = \mathcal{M}_X(t) \mathcal{M}_Y(-t)$. Since $\mathcal{M}_Z(t)$ is a rational function of $t$ as a product of rational functions, do partial fraction decomposition, and read off linear combinations of exponentials and reflected exponentials. – Sasha Nov 26 '11 at 18:26
Thank you for your answer. But as a layman, I'm afraid I become more puzzled by your answer. Is there any textbook or reference on why the probability density of a hypoexponential with parameter $\Theta$ can be written as $f(x)=-\boldsymbol{\alpha}e^{x\Theta}\Theta\boldsymbol{1}$ in the wikipedia article on hypoexponential distribution? – Federico Magallanez Nov 26 '11 at 21:07