Understanding central limit theorem

I am not understanding the central limit theorem.

From wikipedia:

...suppose that a sample is obtained containing a large number of observations, each observation being randomly generated in a way that does not depend on the values of the other observations, and that the arithmetic average of the observed values is computed. If this procedure is performed many times, the central limit theorem says that the computed values of the average will be distributed according to the normal distribution

what I'm confused about is...if we have a sample of n observed values, then the average of the population will be the sum of all the observed values divided by the total number of observed values. So we will have an average....THE average, meaning ONLY one average, so how can ONE value have a "distribution"? Obviously I'm missing something or interpreting what the definition is saying wrongly, so can somebody help me out?

Edit: Should I think of this as like...let's say we have 1 value. It will have an average. Then we have another value, and take the average of the two values. Then a third value, and find the average of the three. Eventually as you get larger and larger numbers, the "distribution" of all these separate averages will be normal, with the average value eventually equaling the expected value mu?

• The key is if this procedure is performed many times. That is, run a bunch of experiments, each with $n$ observations and hence each with their own average. Then look at the distribution of those averages. – aes Dec 4 '14 at 23:02
• is the way I described it in my edit an accurate interpretation of it? @aes – FrostyStraw Dec 4 '14 at 23:05
• No. Think of getting the average of $n$, let's say $n = 100$, observations, calling that your first average. Then go get another 100 observations and take the average, that's your second average. And so on, generate many averages of 100 observations. The distribution of these averages of 100 observations (which is the probability distribution for the average of 100 observations) is your distribution for 100. Then think about different values for $n$. The central limit theorem is a statement about these distributions as $n$ gets large. – aes Dec 4 '14 at 23:15
• so is the distribution of the many averages of 100 observations normal? Or is it possibly not normal, but the distribution of the many averages of 1,000 observations "more normal", and the distribution of the many averages of 10,000 observations "more normal", and so on and so forth? @aes – FrostyStraw Dec 4 '14 at 23:17
• Right, more and more normal as you look at the distributions of averages of more and more observations. – aes Dec 4 '14 at 23:50

You have random variables $X_1,X_2,X_3,\dotsc,X_n$. Let's define a random variable $$Y_n=\frac{X_1,\dotsc,X_n}{n}.$$ Then $Y_n$ will have a normal distribution as $n\to \infty$. $Y_n$ is not the average of averages, it is a random variable that at least takes the values of the averages of $X_1,\dotsc,X_n$.