Does a Chi-Square random variable $\chi^2_1$ mean that only one normal random variable was taken? I'm trying to understand how Chi-Square variables work.
So far, I know that a Chi-Square random variable, $\chi^2$, means that one random value has been taken from a normally distributed graph. Let's say it was the standard normal distribution. This means, $\chi^2$ has a high probability of being zero or near zero.
Here's what I don't understand; How many degrees of freedom does $\chi^2$ have? If it only represents one random variable, then it has zero degrees of freedom, doesn't it?
For example, if I take 5 random variables from a normal distribution, is it $\chi^2_1+\chi^2_1+\chi^2_1+\chi^2_1+\chi^2_1= \chi^2_5$ or is one somehow not counted?
 A: As Robert Israel has pointed out, the sum of squares of $n$ independent random variables with a standard normal distribution has a chi-square distribution with $n$ degrees of freedom.
Take them from a normal distribution whose expectation is $\mu$ and whose standard deviation is $\sigma$, you have have
$$
\left(\frac{X_1-\mu}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-\mu}{\sigma}\right)^2
$$
has chi-square distribution with $n$ degrees of freedom.
So why might it appear that one of them is not counted?  The answer to that comes from such results as this: Suppose instead of the population mean $\mu$, you subtract the sample mean $\overline X$.  Then you have
$$
\left(\frac{X_1-\overline X}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-\overline X}{\sigma}\right)^2,\tag{1}
$$
and this has a chi-square distribution with $n-1$ degrees of freedom.  In particular, if $n=1$, then the sample mean is just the same as $X_1$, so the numerator in the first term is $X_1-X_1$, and the sum is necessarily $0$, so you have a chi-square distribution with $0$ degrees of freedom.
Notice that in $(1)$, you have $n$ terms in the sum, not $n-1$, and they're not independent (since if you take away the exponents, you get $n$ terms that necessarily always add up to $0$) and the standard deviation of the fraction that gets squared is not actually $1$, but less than $1$.  So why does it have the same probability distribution as if there were $n-1$ of them, and they were indepedent, and those standard deviations were each $1$?  The simplest way to answer that may be this:
$$
\begin{bmatrix} X_1 \\  \vdots \\ X_n \end{bmatrix} = \begin{bmatrix} \overline X \\  \vdots \\  \overline X \end{bmatrix} + \begin{bmatrix} X_1 - \overline X \\  \vdots \\  X_n - \overline X \end{bmatrix}
$$
This is the decomposition of a vector into two components orthogonal to each other: one in a $1$-dimensional space and the other in an $n-1$ dimensional space.  Now think about the spherical symmetry of the joint probability distribution, and about the fact that the second projection maps the expected value of the random vector to $0$.
Later edit:
Sometimes it might seem as if two of them are not counted.  Suppose $X_i$ is a normally distributed random variable with expected value $\alpha+\beta w_i$ and variance $\sigma^2$, and they're independent, for $i=1,\ldots,n$.  When $w_i$ is observable and $\alpha$, $\beta$, are not, one may use least-squares estimates $\hat\alpha$, $\hat\beta$.  Then
$$
\left(\frac{X_1-(\alpha+\beta w_1)}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-(\alpha+\beta w_n)}{\sigma}\right)^2 \sim \chi^2_n
$$
but
$$
\left(\frac{X_1-(\hat\alpha+\hat\beta w_1)}{\sigma}\right)^2 + \cdots + \left(\frac{X_n-(\hat\alpha+\hat\beta w_n)}{\sigma}\right)^2 \sim \chi^2_{n-2}.
$$
A similar sort of argument involving orthogonal projections explains this.
One needs these results in order to derive things like confidence intervals for $\mu$, $\alpha$, and $\beta$.
A: If $X_1, \ldots, X_n$ are independent standard normal random variables, then $X_1^2 + \ldots + X_n^2$ has a ChiSquare distribution with $n$ degrees of freedom.
