I'm trying to understand how Chi-Square variables work.

So far, I know that a Chi-Square random variable, $\chi^2$, means that one random value has been taken from a normally distributed graph. Let's say it was the standard normal distribution. This means, $\chi^2$ has a high probability of being zero or near zero.

Here's what I don't understand; How many degrees of freedom does $\chi^2$ have? If it only represents one random variable, then it has zero degrees of freedom, doesn't it?

For example, if I take 5 random variables from a normal distribution, is it $\chi^2_1+\chi^2_1+\chi^2_1+\chi^2_1+\chi^2_1= \chi^2_5$ or is one somehow not counted?

  • $\begingroup$ Yes, $1$ degree of freedom means that you are dealing with the distribution of the square of a standard normal. $\endgroup$ – André Nicolas Nov 16 '12 at 0:43
  • $\begingroup$ I was also taught that you need two standard normals to get one degree of freedom. Do you know what I'm referring to? $\endgroup$ – CodyBugstein Nov 16 '12 at 0:54
  • $\begingroup$ @Imray : That happens when you subtract the sample mean from each of the observations. See my answer below. $\endgroup$ – Michael Hardy Nov 16 '12 at 1:02

As Robert Israel has pointed out, the sum of squares of $n$ independent random variables with a standard normal distribution has a chi-square distribution with $n$ degrees of freedom.

Take them from a normal distribution whose expectation is $\mu$ and whose standard deviation is $\sigma$, you have have $$ \left(\frac{X_1-\mu}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-\mu}{\sigma}\right)^2 $$ has chi-square distribution with $n$ degrees of freedom.

So why might it appear that one of them is not counted? The answer to that comes from such results as this: Suppose instead of the population mean $\mu$, you subtract the sample mean $\overline X$. Then you have $$ \left(\frac{X_1-\overline X}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-\overline X}{\sigma}\right)^2,\tag{1} $$ and this has a chi-square distribution with $n-1$ degrees of freedom. In particular, if $n=1$, then the sample mean is just the same as $X_1$, so the numerator in the first term is $X_1-X_1$, and the sum is necessarily $0$, so you have a chi-square distribution with $0$ degrees of freedom.

Notice that in $(1)$, you have $n$ terms in the sum, not $n-1$, and they're not independent (since if you take away the exponents, you get $n$ terms that necessarily always add up to $0$) and the standard deviation of the fraction that gets squared is not actually $1$, but less than $1$. So why does it have the same probability distribution as if there were $n-1$ of them, and they were indepedent, and those standard deviations were each $1$? The simplest way to answer that may be this: $$ \begin{bmatrix} X_1 \\ \vdots \\ X_n \end{bmatrix} = \begin{bmatrix} \overline X \\ \vdots \\ \overline X \end{bmatrix} + \begin{bmatrix} X_1 - \overline X \\ \vdots \\ X_n - \overline X \end{bmatrix} $$ This is the decomposition of a vector into two components orthogonal to each other: one in a $1$-dimensional space and the other in an $n-1$ dimensional space. Now think about the spherical symmetry of the joint probability distribution, and about the fact that the second projection maps the expected value of the random vector to $0$.

Later edit: Sometimes it might seem as if two of them are not counted. Suppose $X_i$ is a normally distributed random variable with expected value $\alpha+\beta w_i$ and variance $\sigma^2$, and they're independent, for $i=1,\ldots,n$. When $w_i$ is observable and $\alpha$, $\beta$, are not, one may use least-squares estimates $\hat\alpha$, $\hat\beta$. Then $$ \left(\frac{X_1-(\alpha+\beta w_1)}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-(\alpha+\beta w_n)}{\sigma}\right)^2 \sim \chi^2_n $$ but $$ \left(\frac{X_1-(\hat\alpha+\hat\beta w_1)}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-(\hat\alpha+\hat\beta w_n)}{\sigma}\right)^2 \sim \chi^2_{n-2}. $$ A similar sort of argument involving orthogonal projections explains this.

One needs these results in order to derive things like confidence intervals for $\mu$, $\alpha$, and $\beta$.


If $X_1, \ldots, X_n$ are independent standard normal random variables, then $X_1^2 + \ldots + X_n^2$ has a ChiSquare distribution with $n$ degrees of freedom.

  • $\begingroup$ Todah but I was taught that the degrees of freedom = $n-1$. Too many contradictions :) $\endgroup$ – CodyBugstein Nov 16 '12 at 0:48
  • $\begingroup$ No contradictions. Read my comment under the question and my answer. $\endgroup$ – Michael Hardy Nov 16 '12 at 3:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.