# What is an intuitive meaning of $E(\overline { X } )$ and $Var(\overline { X } )$?

Let $X$ be a random variable distributed over, for example say, the Binomial Distribution. Then $P(X)$ is the probability of getting $x$ successful trials in $n$ total trials.

So I saw a notation that represents the mean of random variables that made me I feel sceptical about my understanding of all the notations I have known. So here's my understanding of the notations:

When it says the expectation of $X$, $E(X)$, does it mean over a long run, $E(X)$ is the likely number of successful trials? In other words, the expected value of $X$ is the expected number of successful trials we would expect in a long run?

When it says the variance of $X$, $Var(X)$, does it mean how spread out the probability of successful trials are? Like how far apart the probability between the successful trials are?

Now, here's the confusing part. I see a notation like this: $\overline { X } =\frac { 1 }{ n } \sum _{ i=1 }^{ n }{ { X }_{ i } }$ and this is called the mean of all the random variables. But it doesn't seem to make sense to me. $X$ is the random variable and carries the value that is the number of successful trials. The average of $X$ is like the average number of successful trials?

Does it then mean $\overline { X } =E(X)$?

Then, there is also the expectation of the mean of all the random variables, $E(\overline { X } )$. So does this represent the average of the average of all the random variables, which means $E(\overline { X } )=E(E(X))$? But at this point, I couldn't understand what it means intuitively. What does it mean here to say the average of the average of all random variables?

Similarly, $Var(\overline { X } )$ is also a confusing term to me. Since $\overline { X }$ is just the average value, what spread does it have?

What is the intuitive meaning of this $\overline { X }$ mean of all random variables $X$ and what does this add on to the meaning of $E(\overline { X } )$ and $Var(\overline { X } )$?

• $\bar{X}$ is the average of a set of random variables, so is a variable itself (i.e. it depends on the particular values of the $X_i$ for any trial). Things like $E(X_i)$ are independent of any particular value for the $X_i$ but depend instead on the probability distribution for $X_i$. Commented Feb 19, 2012 at 10:03
• Just adding to @AlexBecker's comment: In the discrete case, $E(X)$ is the mean over the whole sample space. It is a priori, and depends on an assumed probability distribution for the $X_i$. A given mean $\overline{X}$, however, is a posteriori, depending on a sample of values $X_1,\dots,X_n$. Commented Feb 19, 2012 at 10:23
• You should have written $P(X=x)$ is the probability of getting $x$ successes in $n$ trials. "$P(X)$" doesn't even mention $x$. Commented Feb 19, 2012 at 18:24
• To rephrase the previous comments a bit, $E(X)$ is the average with respect to the probability measure, and $\text{Var}(X)$ is the average square of deviation from $E(X)$ with respect to the probability measure. Probability measure is something that in a sense tells you how to take meaningful averages. Commented Feb 20, 2012 at 3:04

You have a vast population of people who have different heights, and you choose one at random. That person's height is $X$. $E(X)$ is the average height of everyone in the population. Then you pick $20$ people at random. Their heights are $X_1,\ldots,X_{20}$. Their average height is $\bar{X} = (X_1+\cdots+X_{20})/20$. That is a random variable because if you pick another set of $20$, it has a different value---thus it varies randomly. The expected value $E(\bar{X})$ is the same as the expected value $E(X)$. But the variance $\operatorname{var}(\bar{X})$ is smaller than the variance $\operatorname{var}(X)$, because on average one sample of $20$ differs less from another sample of $20$ than one individual differs from another individual.

• Can I say that $X_1, \cdots ,X_20$ are actually numbers on its own. It is a random variable only because these group of people were randomly sampled. But this wouldn't this also mean that $X_1, \cdots ,X_20$ are actually the $\omega$ inside the population of $\Omega$? Also, since $E(\bar{X})=\bar{X} = (X_1+\cdots+X_{20})/20$ are sampled values, how would the $E(\bar{X})$ same as $E(X)$, which consists of the whole population's height which may make the mean value very different from the sampled people's height? Commented Feb 20, 2012 at 4:21
• I didn't say $\bar{X}$ is the same as $E(X)$. That would be false. I said $E(\bar{X})$ is the same as $E(X)$. Think about the difference. Commented Feb 20, 2012 at 18:33
• You wrote "since $E(\bar{X}) = \bar{X}$. That's not true at all. Commented Feb 20, 2012 at 18:33
• Just to be explicit: $\bar{X}$ is based on only $20$ values, but $E(\bar{X})$ is based on all possible samples of $20$ from the whole population. Commented Feb 20, 2012 at 19:48

You should think of $E(X)$ and $Var(X)$ (when they exist) as parameters which describe the distribution of the random variable $X$. They are numbers. They are not random variables. So, although you can write $E(E(X))$, it's not very useful since the expected value of a constant is just the constant. So you have $E(E(X)) = E(X)$.

Sometimes it is helpful to call $E(X)$ the "population average."

Now, you should think of $\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i$ as simply a function of $n$ random variables, albeit a very useful one. You shouldn't equate it in your mind with the population average $E(X)$ (a number). Rather $\bar{X}_n$ is a random variable.

Given some outcome $\omega$, you observe the sample values $x_1 = X_1(\omega), \dots, x_n = X_n(\omega)$ and you can compute the so called "sample average," which is a particular realization of $\bar{X}_n$, namely

$$\bar{X}_n(\omega) = \frac{1}{n}\sum_{i=1}^n X_i(\omega) = \frac{1}{n}\sum_{i=1}^n x_i$$

Now, this sample average is a number, but it could be quite different from the population average, the number $E(X)$. However, the Law of Large Numbers says that if $n$ is large enough then (any realization of) $\bar{X}_n$ will be close to $E(X)$.

And, yes, you can take the expected value of the random variable $\bar{X}_n$, as you suggest, and you will simply have

$$E(\bar{X}_n) = \frac{1}{n}\sum_{i=1}^n E(X_i) = E(X)$$

(The last equality assumes the $X_i$ all come from a distribution with mean $E(X)$.)

As for the variance, no, $Var(X)$ does not measure the spread between successes in repeated trials. Rather, it is a measure of the spread of the possible values of the function $X$ about its mean $E(X)$. If the variance is larger (smaller) that means there is a larger (smaller) chance $X$ will take on a value far from $E(X)$. In your example, the possible values are 0 and 1, with probabilities $(1-p)$ and $p$, say. The variance would be much bigger if you chose, say, 0 and 100 as the possible values for $X$.

Finally, note that the variance of $\bar{X}_n$ gets smaller as $n$ gets larger. This is telling you that the larger $n$ is, the more likely the observed value of $\bar{X}_n$ will be close to $E(X)$.

• Small nitpick: for the SLLN the $n$ we need to take depends on $\omega$ whereas when you say "if $n$ is large enough then (for any realization of)..." it makes it sound as if we can choose $n$ which works uniformly for all $\omega$. It also doesn't hit the fact that it doesn't work for all $\omega$, but almost all $\omega$ but I'm not sure how to say this at the appropriate level for OP.
– guy
Commented Feb 19, 2012 at 15:05
• I think you're using the word "parameter" too loosely. When you have a parametrized family of probability distributions, you have parameters. But $E(X)$ can exist regardless of anything about parametrized families. You don't have to think about more than one distribution---let alone a parametrized family of distributions---to speak of $E(X)$. Commented Feb 19, 2012 at 18:26
• @MichaelHardy I disagree. Even if we're speaking of a single distribution, if $E(X)$ exists, I see no problem with calling it a parameter of the distribution of $X$. I suppose the term could be defined very narrowly, reserved for only those parameters which uniquely identify a distribution within a parametrized family, but that is too narrow, imho. I prefer to take parameter to mean "feature of a distribution." (For example, I would say the moments of a chi-square are parameters of the distribution, even though they are not the parameter we use to identify the distribution.) Commented Mar 24, 2012 at 13:39

The distinction among $$\bar X$$, $$E(\bar X)$$ and $$E(X)$$ is indeed not obvious and in my personal oppinion, often not well clarified in introductory statistics courses. The same is valid for $$V(X)$$ and $$S^2$$ (variance from a sample).

Let's use your suggestion and consider a $$X$$ r.v. Binomial(n,p). For any discrete r.v. $$X$$, if $$E(X)$$ and $$V(X)$$ exist, $$E(X)=\sum_{x\in \Omega_x} x P(X=x)\ \text{and}\ \ V(X)=E[(X-E(X))^2]$$ in which $$\Omega_x$$ is the set of possible values for $$X$$. You can understand $$E(X)$$ and $$V(X)$$ as theoretical constants associated with the PMF. For the Binomial$$(n,p)$$, $$E(X)=n p$$ and $$V(X)=np(1-p)$$, constants that depend on the parameters $$n$$ and $$p$$ from the Binomial. You can regard them as the theoretical mean and theoretical variance associated with the Binomial(n,p).

Now consider an independent sample size $$m$$ from this variable $$X$$ Binomial$$(n,p)$$ given by $$X_1,X_2,\ldots,X_m$$ and compute the average and the sample variance using: $$\bar X_m=\dfrac{\sum_{i=1}^{m} X_i}{m}\ \ \text{and}\ \ S^2=\dfrac{\sum_{i=1} (X_i-\bar X_m)^2}{m-1}$$

Because of an important property of the operator $$E(\cdot)$$ it is true for any r.v.s $$X$$ and $$Y$$ and constants $$a$$ and $$b$$ that $$E(aX+bY)=aE(X)+bE(Y)$$.

Using this property we can compute $$E(\bar X_m)$$: $$E(\bar X_m)=\dfrac{\sum_{i=1}^{m} E(X_i)}{m}=\dfrac{m E(X)}{m}=E(X),$$ as for any $$X_i$$ from the sample, $$E(X_i)=E(X)$$. With some additional effort we can also show that $$E(S^2)=V(X).$$

This is the reason why $$E(\bar X_m)=E(X)$$. And by the Law of Large Numbers (LLN) it is true that (under regularity conditions) $$\bar X_m$$ converges to $$E(X)$$ as $$m\rightarrow \infty$$ by statistical notions. By the same reason $$S^2$$ converges to $$V(X)$$ as $$m\rightarrow \infty$$.

Let's return to our Binomial$$(n,p)$$ with $$E(X)=n p$$ and $$V(X)=np(1-p)$$ and do some experimentation. If, for instance, $$n=10$$ and $$p=0.3$$, we have $$E(X)=10\times 0.3=3$$ and $$V(X)=10\times 0.3\times 0.7=2.1$$

Using a statistical software, let us obtain 2 samples size m=5 from this Binomial$$(n=10,p=0.3)$$ (using for instance rbinom$$(5,10,0.3)$$ in software R):

1. Sample 1: 1,2,2,7,4 with $$\bar X_5=3.2$$ and $$S^2=5.7$$
2. Sample 2: 4,4,1,1,1 with $$\bar X_5=2.2$$ and $$S^2=2.7$$

Notice that, in both samples, $$\bar X_5$$ and $$S^2$$ are not equal $$E(X)=3$$ and $$V(X)=2.1$$. The values will tend to be closer as the sample size increases, as theoreticaly expected by the LLN.