The intuition behind gamma distribution What is the intuition behind gamma distribution?
For instance, I understand how to "construct" Gaussian distribution. This is my intuition:


*

*Bernoulli distribution - which is simple concept

*A sequence of Bernoulli trials is a Binomial distribution. I understand how binomial coefficient is constructed

*Using Stirling approximation we can deduce Gaussian distribution


Hence I understand that shape of a Gaussian distribution is determined by the binomial coefficients and so on.
How can Gamma distribution be derived step by step using relatively simple concepts?
 A: The short answer is "prove the pdf is non-negative & integrates to 1", since that guarantees we're dealing with a probability distribution. But if you're looking for a motivation for why anything would be Gamma-distributed, here's the idea.
The $\alpha =1$ special case, with pdf $e^{-\beta x}$ for $x>0$, is the so-called exponential distribution, which concerns the lifetime of something that has the same risk of failure in a given time period regardless of its history (we call this memorylessness), e.g. a radioactive atom. (The mean lifetime is $\beta^{-1}$.) If $\alpha$ is a positive integer, you have the distribution obtained by summing $\alpha$ exponential iids. In general $\alpha$ can be any positive real, but that's just a continuous generalisation.
A: I understand the exponential distribution in a following way:
$$e^{-\lambda \cdot x} = \lim_{N \to \infty} (1 - \frac{\lambda}{N})^{N \cdot x}$$
Where:
$\lambda$ - number of events per one time unit (denoted by $1$)
$N$ - tends to infinity. It splits whole time unit into $N$ small intervals each of a length of $\frac{1}{N}$, such that only one event can occur within this small interval
$\frac{\lambda}{N}$ - is a probability of an event within one small time frame. Each time frame is a Bernoulli trial: event - success, no event - failure.
$(1 - \frac{\lambda}{N})$ - probability of "failure" - no event
$N \cdot x$ - is a number of consecutive "failures". Where $x$ is a part of 1 interval, let $N = 1000$ - some big number, then half of the interval ($x = 0.5$) would be $N \cdot x = 1000 * 0.5 = 500$ small time frames
$(1 - \frac{\lambda}{N})^{N \cdot x}$ - probability of $N \cdot x$ consecutive failures, or probability that the event will not occur $x$ amount of time
A: The following is some intuition behind the formula.
The Gamma distribution answer the question: 'How long do we have to wait between $a$ events with exponential wait times of $\frac{1}{B}e^{-t/B}$?'
For $a=1$ it's distributed as $\frac{1}{B}e^{-t/B}$, we already know that, it's just the exponential distribution.
How long will we have to wait for two events? In other words, for $a=2$?
Let's talk about the likleyhood of waiting $5$ seconds ($T=5$) if we have to wait for two exponential events ($a=2$)
Roughly $P(T=5| a=2) = \sum_0^5 P(T = i| a=1)P(T = 5-i|a=1)$
We can slice time into smaller slices until the above is an integral.
$p(t=5) =\int_0^5 p(T = i | a=1)p(T = 5-i| a=1)di = \frac{1}{B^2}\int_0^5 e^{-i/B}e^{(-5-i)/B} di=\frac{1}{B^2}\int_0^5 e^{-5/B}  di=  \frac{1}{B^2}e^{-5/B} \int_0^5 di = 5\frac{1}{B^2}e^{-5/B} $
replacing $5$ with '$x$' we get:
$p(t=x|a=2) = x\frac{1}{B^2}e^{-x/B}$
Let's now increase to $a = 3$
In the following the double integral is over the area $i+j < 5$
$p(t=5) = \int \int p(T = i| a=1 )p(T = j| a=1)p(T=5-i-j| a=1)didj = \frac{1}{B^3}\int \int e^{-i/B}e^{-j/B}e^{-(5-i-j)/B} didj=\frac{1}{B^3} e^{-5/B} \int\int didj=  \frac{1}{B^3}e^{-5/B} \int\int didj = 12.5\frac{1}{B^3}e^{-5/B} $
Where $12.5=\frac{5^2}{2}$ is the area of the integration triangle with side lengths 5.
replacing $5$ with $x$ we get:
$p(t=x|a=3) = \frac{x^2}{2}\frac{1}{B^3}e^{-x/B}$
As we increase to higher $a$'s we will be integrating over simplexs of dimension $a-1$.
Therefore:
$p(t=x|a=n) = \frac{x^{n-1}}{(n-1)!}\frac{1}{B^n}e^{-x/B} = \frac{x^{n-1}}{\Gamma(n-1)B^n}e^{-x/B}$
