# Does a Chi-Square random variable $\chi^2_1$ mean that only one normal random variable was taken?

I'm trying to understand how Chi-Square variables work.

So far, I know that a Chi-Square random variable, $\chi^2$, means that one random value has been taken from a normally distributed graph. Let's say it was the standard normal distribution. This means, $\chi^2$ has a high probability of being zero or near zero.

Here's what I don't understand; How many degrees of freedom does $\chi^2$ have? If it only represents one random variable, then it has zero degrees of freedom, doesn't it?

For example, if I take 5 random variables from a normal distribution, is it $\chi^2_1+\chi^2_1+\chi^2_1+\chi^2_1+\chi^2_1= \chi^2_5$ or is one somehow not counted?

-
Yes, $1$ degree of freedom means that you are dealing with the distribution of the square of a standard normal. –  André Nicolas Nov 16 '12 at 0:43
I was also taught that you need two standard normals to get one degree of freedom. Do you know what I'm referring to? –  Imray Nov 16 '12 at 0:54
@Imray : That happens when you subtract the sample mean from each of the observations. See my answer below. –  Michael Hardy Nov 16 '12 at 1:02

As Robert Israel has pointed out, the sum of squares of $n$ independent random variables with a standard normal distribution has a chi-square distribution with $n$ degrees of freedom.

Take them from a normal distribution whose expectation is $\mu$ and whose standard deviation is $\sigma$, you have have $$\left(\frac{X_1-\mu}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-\mu}{\sigma}\right)^2$$ has chi-square distribution with $n$ degrees of freedom.

So why might it appear that one of them is not counted? The answer to that comes from such results as this: Suppose instead of the population mean $\mu$, you subtract the sample mean $\overline X$. Then you have $$\left(\frac{X_1-\overline X}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-\overline X}{\sigma}\right)^2,\tag{1}$$ and this has a chi-square distribution with $n-1$ degrees of freedom. In particular, if $n=1$, then the sample mean is just the same as $X_1$, so the numerator in the first term is $X_1-X_1$, and the sum is necessarily $0$, so you have a chi-square distribution with $0$ degrees of freedom.

Notice that in $(1)$, you have $n$ terms in the sum, not $n-1$, and they're not independent (since if you take away the exponents, you get $n$ terms that necessarily always add up to $0$) and the standard deviation of the fraction that gets squared is not actually $1$, but less than $1$. So why does it have the same probability distribution as if there were $n-1$ of them, and they were indepedent, and those standard deviations were each $1$? The simplest way to answer that may be this: $$\begin{bmatrix} X_1 \\ \vdots \\ X_n \end{bmatrix} = \begin{bmatrix} \overline X \\ \vdots \\ \overline X \end{bmatrix} + \begin{bmatrix} X_1 - \overline X \\ \vdots \\ X_n - \overline X \end{bmatrix}$$ This is the decomposition of a vector into two components orthogonal to each other: one in a $1$-dimensional space and the other in an $n-1$ dimensional space. Now think about the spherical symmetry of the joint probability distribution, and about the fact that the second projection maps the expected value of the random vector to $0$.

Later edit: Sometimes it might seem as if two of them are not counted. Suppose $X_i$ is a normally distributed random variable with expected value $\alpha+\beta w_i$ and variance $\sigma^2$, and they're independent, for $i=1,\ldots,n$. When $w_i$ is observable and $\alpha$, $\beta$, are not, one may use least-squares estimates $\hat\alpha$, $\hat\beta$. Then $$\left(\frac{X_1-(\alpha+\beta w_1)}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-(\alpha+\beta w_n)}{\sigma}\right)^2 \sim \chi^2_n$$ but $$\left(\frac{X_1-(\hat\alpha+\hat\beta w_1)}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-(\hat\alpha+\hat\beta w_n)}{\sigma}\right)^2 \sim \chi^2_{n-2}.$$ A similar sort of argument involving orthogonal projections explains this.

One needs these results in order to derive things like confidence intervals for $\mu$, $\alpha$, and $\beta$.

-

If $X_1, \ldots, X_n$ are independent standard normal random variables, then $X_1^2 + \ldots + X_n^2$ has a ChiSquare distribution with $n$ degrees of freedom.

-
Todah but I was taught that the degrees of freedom = $n-1$. Too many contradictions :) –  Imray Nov 16 '12 at 0:48
No contradictions. Read my comment under the question and my answer. –  Michael Hardy Nov 16 '12 at 3:59