Chi square and gamma distribution.

Question

I have studied the intuiton of the gamma distribution and have understood the following:

Let us suppose that we want to study the probability of waiting time until the $\alpha$-th event occurs. Let $W$ be the corresponding ramdon value. Then we want to calculate $F(w)=P(W\leq w)=1-P(W>w)$.

Now, the waiting time $W$ is greater than some value $w$ only if there are fewer than $α$ events in the interval $[0,w]$. That is:

$$ \begin{align} F(w) & = 1 − P(\text{fewer than $α$ events in $[0,w]$)} \\ & = 1 − P(\text{0 events or 1 event or … or $(α−1)$ events in $[0,w]$)} \end{align} $$

Then one can use the Poisson distribution with mean $\lambda w$ to obtain

$$F(w)=1-\sum\limits_{k=0}^{\alpha-1} \dfrac{(\lambda w)^k e^{-\lambda w}}{k!}$$

Calculating the first derivation and doing some simplifications with $\lambda=1/\theta$, this leads to $$f(w)=\frac 1 {\Gamma(\alpha) \theta^\alpha} e^{-w/\theta} w^{\alpha-1}$$

where $\alpha$ can now be used as a real positive number.

So in a way the Gamma distributions gives the posibility of that $\alpha$ events occur until some determined time. Thus, $\theta$ is the mean waiting time until the first event, and $\alpha$ is the number of events for which you are waiting to occur.

That I would like to know it the following:

Is there any geometric/intuitive way to know why the Chi squared distriution takes the values $\theta=2$ and $\alpha=r/2$ as a special case of the gamma distribution? What does make these values special and important/interesting?

If this is not the right way to lighten the Chi square distribution: Is there a geometric/intuitive way to understand the Chi Square distribution as a sum of squares from normal distributed ramdom variables? Why Is the sum of this squares interesting/special?

Maybe there are more technical than geometrical reasons for the approach of the Chi square distribution, but I would feel more confortable to gain an inside of what this distributions does insteed of where it can be use for...

Thank you in advance for any help!

Answer 1

The chi-squared family is indeed a subfamily of gamma, but some of the intuition and motivation is different. I'll try to give a few examples.

If $X_1, X_2, \dots, X_n$ is a random sample from standard normal ($\mathsf{Norm}(0,1)$), then $Q = \sum_{i=1}^n X_i^2 \sim \mathsf{Chisq}(n).$ Considering $(X_1, X_2, \dots, X_n)$ as a vector in $n$-space from the origin to the data point, $Q$ is the squared length of the vector (by a multi-dimensional version of the Pythagorean Theorem.) Such points are distributed according to an $n$-variate uncorrelated standard normal distribution.

Thus $P(Q \le 1)$ is the probability that the data point is inside an $n$-sphere of hyper-radius 1 centered at the origin. The amount of 'room' in $n$-space is so great that it is difficult to imagine. As dimensionality increases, the proportion of our multivariate normal points within one unit of the origin decreases rapidly. Below are plots PDFs of $\mathsf{Chisq}(n),$ for $n = 2, 4, 8, 16.$ The area to the left of the vertical red line in each plot represents the proportion of multivariate normal data points within one unit of the origin.

Here is a table (using R statistical software) of $P(Q \le 1)$ for several values of $n.$

n = 1:12;  prob = pchisq(1, n)
round(cbind(n, prob), 4)
     n   prob
 ##  1 0.6827  # Equivalent to P(-1 < Z < 1) = .6827, for Z ~ NORM(0,1)
 ##  2 0.3935
 ##  3 0.1987
 ##  4 0.0902
 ##  5 0.0374
 ##  6 0.0144
 ##  7 0.0052
 ##  8 0.0018
 ##  9 0.0006
 ## 10 0.0002
 ## 11 0.0001
 ## 12 0.0000

A consequence is that it takes a huge amount of data to study the behavior near the origin of an $n$-variate normal model for large $n.$ Such difficulties are called the "curse of dimensionality."

As mentioned above chi-squared distributions are a sub-family of gamma distributions: $\mathsf{Chisq}(n) \equiv \mathsf{Gamma}(shape = n/2, rate = 2).$ Also, $\mathsf{Chisq}(2) \equiv \mathsf{Gamma}(shape = 1, rate = 2) \equiv \mathsf{Exp}(rate = 2).$ If $Q \sim \mathsf{Chisq}(n),$ then $E(Q) = n$ and $Var(Q) = 2n.$

Notes: The parameter $n$ is often called 'degrees of freedom' and often matches the dimensionality of the space or subspace of a model. (The parameter is sometimes written as $\nu$, Greek letter nu.) The chi in chi-squared is the Greek letter $\chi$ pronounced in the US to rhyme with by, and with a hard k-sound as in other words of Greek origin: chorus, Christimas, chromatic, and so on. Elsewhere, the pronunciation is sometimes like English key. The distribution name is sometimes written as $\chi^2.$