# Question

Let $$S_k=\sum_{i=1}^k X_i$$ be the sum of $k$ independent random variables. I am interested in closed-form expressions of the pdf of $S_k$.

In general, the pdf is given by the $k$-fold convolution of the individual probability distributions such that $$f_S=(f_1*\ldots*f_k)(s),$$ where $f_i$ is the probability distribution of the $i^{\rm th}$ random variable.

In particular, $S_k$ is gamma-distributed if the individual random variables are gamma-distributed with the same scale parameter.

What other distributions exist that satisfy the same property but have at least two parameters?

# Motivation

I am performing simulations involving a sum of random variables. Performing the convolutions numerically is too expensive computationally, which is why I would like to find out more about distributions that are "self-repeating" under convolution.

# Some thoughts

Formally, we can define the characteristic function $\hat{f}(k;{\bf a})$ of a probability distribution $f(x;{\bf a})$, where $\bf a$ is a set of parameters characterising the distribution.

By the convolution theorem, the product of the characteristic functions corresponds to the convolution of the probability distributions.

The family of distributions I am interested in thus satisfies $$\hat{f}(k;{\bf a})\times \hat{f}(k;{\bf b}) = \hat{f}(k;g({\bf a},{\bf b})),$$ where $g$ is an arbitrary function that is symmetric with respect to exchange of $\bf a$ and $\bf b$.

Unfortunately, this formalism has not helped me come up with an appropriate family of distributions.

-
Two posted answers begin by mentioning infinite divisibility, but that is not essential, given the way the question is phrased. See also my answer below. – Michael Hardy Feb 10 '13 at 18:32

You might want to look into infinite divisibility. Many probability distributions are infinitely divisible, which is easy to recognize from their characteristic function. Among them, the stable distributions have two parameters, scale and location.

Examples are normal distribution, Cauchy distribution, Lévy distribution, and $\alpha$-stable distributions.

-
Thanks for the hint. I found this book, which discusses the topic in more detail: amazon.com/… – Till Hoffmann Feb 10 '13 at 18:36
Infinite divisibility really isn't needed to answer this question. Some families of distributions that are not infinitely divisible can serve very well. – Michael Hardy Feb 10 '13 at 18:39
@MichaelHardy true. But he wanted something with two parameters. So as a step towards stable-distributions, I chose to introduce it. – Memming Feb 10 '13 at 18:53
The binomial and negative binomial families each have two parameters. – Michael Hardy Feb 10 '13 at 18:55
@MichaelHardy But one of them has to be fixed to stay within the same distribution family, right? – Memming Feb 10 '13 at 22:31

There are many distributions that fit the bill. These are families of infinitely divisible distributions, closed under convolution. Examples:

to name a few.

-

Two other answers mention infinite divisibility, but that's not needed. The list given in Sasha's and Memming's answers are good as far as they go, but we can add some distributions that are not infinitely divisible.

The family of binomial distributions is closed under convolution of probability mass functions, and is not infinitely divisible. If $X\sim\operatorname{Bin}(n,p)$ and $Y\sim\operatorname{Bin}(m,p)$ and these are independent, then $X+Y\sim\operatorname{Bin}(n+m,p)$.

The negative binomial distributions are also closed under convolution. Here we need to attend to an issue of conventions. To say $X$ is negative-binomially distributed with parameters $n$, $p$ could mean

• by one convention, that $X$ is the number of independent Bernoulli trials needed to get $n$ successes, with probability $p$ of success on each trial; or
• by another convention, that $X$ is the number of failures before the $n$th success in independent Bernoulli trials with probability $p$ of success on each trial.

By the first convention, the distribution is supported on the set $\{n,n+1,n+2,\ldots\}$.

By the second convention, the distribution is supported on the set $\{0,1,2,\ldots\}$.

By either convention, it is closed under convolution.

By the second convention, it is infinitely divisible; by the first, it is not.

-

It seems the only closed-form expressions of such pdf's have a single parameter. The Levy jump-diffusion stochastic process adds a skew but crossing-time is not an analytic expression. This implies you need a mixture method or similar to obtain a second parameter without heavy numerical integration/simulation/convolution.

Since Exponential families have conjugate priors the family is closed under Bayesian updating - this insight may be useful in adding an extra "uncertainty" parameter.

Certain mixtures e.g. the Generalised Hyperbolic families are also closed. In that family it seems NIG (normal-inverse Gaussian distribution) is studied for modelling jump-diffusion.

## Bernoulli discrete event discrete time

• hazard / inter-arrival time: geometric distribution
• arrival of number of events : negative binomial distribution
• number in an interval : binomial distribution
• conjugate prior on p is Beta

## Poisson discrete event continuous time

• hazard / inter-arrival time: exponential distribution
• arrival of number of events: gamma distribution
• number in an interval : Poisson distribution

## Wiener (random walk) continuous stochastic process

• no infinitesimal 'event'
• first passage time to 'a': $f(t) = {ae^{−a^2/2t}\over\sqrt{2\pi t^3}}$ don't know a name
• value at a given time : normal distribution
-