Let $X$ be a random variable distributed over, for example say, the Binomial Distribution. Then $P(X)$ is the probability of getting $x$ successful trials in $n$ total trials.

So I saw a notation that represents the mean of random variables that made me I feel sceptical about my understanding of all the notations I have known. So here's my understanding of the notations:

When it says the expectation of $X$, $E(X)$, does it mean over a long run, $E(X)$ is the likely number of successful trials? In other words, the expected value of $X$ is the expected number of successful trials we would expect in a long run?

When it says the variance of $X$, $Var(X)$, does it mean how spread out the probability of successful trials are? Like how far apart the probability between the successful trials are?

Now, here's the confusing part. I see a notation like this: $\overline { X } =\frac { 1 }{ n } \sum _{ i=1 }^{ n }{ { X }_{ i } } $ and this is called the mean of all the random variables. But it doesn't seem to make sense to me. $X$ is the random variable and carries the value that is the number of successful trials. The average of $X$ is like the average number of successful trials?

Does it then mean $\overline { X } =E(X)$?

Then, there is also the expectation of the mean of all the random variables, $E(\overline { X } )$. So does this represent the average of the average of all the random variables, which means $E(\overline { X } )=E(E(X))$? But at this point, I couldn't understand what it means intuitively. What does it mean here to say the average of the average of all random variables?

Similarly, $Var(\overline { X } )$ is also a confusing term to me. Since $\overline { X } $ is just the average value, what spread does it have?

What is the intuitive meaning of this $\overline { X } $ mean of all random variables $X$ and what does this add on to the meaning of $E(\overline { X } )$ and $Var(\overline { X } )$?

  • 2
    $\begingroup$ $\bar{X}$ is the average of a set of random variables, so is a variable itself (i.e. it depends on the particular values of the $X_i$ for any trial). Things like $E(X_i)$ are independent of any particular value for the $X_i$ but depend instead on the probability distribution for $X_i$. $\endgroup$ – Alex Becker Feb 19 '12 at 10:03
  • $\begingroup$ Just adding to @AlexBecker's comment: In the discrete case, $E(X)$ is the mean over the whole sample space. It is a priori, and depends on an assumed probability distribution for the $X_i$. A given mean $\overline{X}$, however, is a posteriori, depending on a sample of values $X_1,\dots,X_n$. $\endgroup$ – bgins Feb 19 '12 at 10:23
  • $\begingroup$ You should have written $P(X=x)$ is the probability of getting $x$ successes in $n$ trials. "$P(X)$" doesn't even mention $x$. $\endgroup$ – Michael Hardy Feb 19 '12 at 18:24
  • $\begingroup$ To rephrase the previous comments a bit, $E(X)$ is the average with respect to the probability measure, and $\text{Var}(X)$ is the average square of deviation from $E(X)$ with respect to the probability measure. Probability measure is something that in a sense tells you how to take meaningful averages. $\endgroup$ – Alexei Averchenko Feb 20 '12 at 3:04

You have a vast population of people who have different heights, and you choose one at random. That person's height is $X$. $E(X)$ is the average height of everyone in the population. Then you pick $20$ people at random. Their heights are $X_1,\ldots,X_{20}$. Their average height is $\bar{X} = (X_1+\cdots+X_{20})/20$. That is a random variable because if you pick another set of $20$, it has a different value---thus it varies randomly. The expected value $E(\bar{X})$ is the same as the expected value $E(X)$. But the variance $\operatorname{var}(\bar{X})$ is smaller than the variance $\operatorname{var}(X)$, because on average one sample of $20$ differs less from another sample of $20$ than one individual differs from another individual.

  • $\begingroup$ Can I say that $X_1, \cdots ,X_20$ are actually numbers on its own. It is a random variable only because these group of people were randomly sampled. But this wouldn't this also mean that $X_1, \cdots ,X_20$ are actually the $\omega$ inside the population of $\Omega$? Also, since $E(\bar{X})=\bar{X} = (X_1+\cdots+X_{20})/20$ are sampled values, how would the $E(\bar{X})$ same as $E(X)$, which consists of the whole population's height which may make the mean value very different from the sampled people's height? $\endgroup$ – xenon Feb 20 '12 at 4:21
  • $\begingroup$ I didn't say $\bar{X}$ is the same as $E(X)$. That would be false. I said $E(\bar{X})$ is the same as $E(X)$. Think about the difference. $\endgroup$ – Michael Hardy Feb 20 '12 at 18:33
  • $\begingroup$ You wrote "since $E(\bar{X}) = \bar{X}$. That's not true at all. $\endgroup$ – Michael Hardy Feb 20 '12 at 18:33
  • 1
    $\begingroup$ Just to be explicit: $\bar{X}$ is based on only $20$ values, but $E(\bar{X})$ is based on all possible samples of $20$ from the whole population. $\endgroup$ – Michael Hardy Feb 20 '12 at 19:48

You should think of $E(X)$ and $Var(X)$ (when they exist) as parameters which describe the distribution of the random variable $X$. They are numbers. They are not random variables. So, although you can write $E(E(X))$, it's not very useful since the expected value of a constant is just the constant. So you have $E(E(X)) = E(X)$.

Sometimes it is helpful to call $E(X)$ the "population average."

Now, you should think of $\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i$ as simply a function of $n$ random variables, albeit a very useful one. You shouldn't equate it in your mind with the population average $E(X)$ (a number). Rather $\bar{X}_n$ is a random variable.

Given some outcome $\omega$, you observe the sample values $x_1 = X_1(\omega), \dots, x_n = X_n(\omega)$ and you can compute the so called "sample average," which is a particular realization of $\bar{X}_n$, namely

$$\bar{X}_n(\omega) = \frac{1}{n}\sum_{i=1}^n X_i(\omega) = \frac{1}{n}\sum_{i=1}^n x_i$$

Now, this sample average is a number, but it could be quite different from the population average, the number $E(X)$. However, the Law of Large Numbers says that if $n$ is large enough then (any realization of) $\bar{X}_n$ will be close to $E(X)$.

And, yes, you can take the expected value of the random variable $\bar{X}_n$, as you suggest, and you will simply have

$$E(\bar{X}_n) = \frac{1}{n}\sum_{i=1}^n E(X_i) = E(X)$$

(The last equality assumes the $X_i$ all come from a distribution with mean $E(X)$.)

As for the variance, no, $Var(X)$ does not measure the spread between successes in repeated trials. Rather, it is a measure of the spread of the possible values of the function $X$ about its mean $E(X)$. If the variance is larger (smaller) that means there is a larger (smaller) chance $X$ will take on a value far from $E(X)$. In your example, the possible values are 0 and 1, with probabilities $(1-p)$ and $p$, say. The variance would be much bigger if you chose, say, 0 and 100 as the possible values for $X$.

Finally, note that the variance of $\bar{X}_n$ gets smaller as $n$ gets larger. This is telling you that the larger $n$ is, the more likely the observed value of $\bar{X}_n$ will be close to $E(X)$.

  • $\begingroup$ Small nitpick: for the SLLN the $n$ we need to take depends on $\omega$ whereas when you say "if $n$ is large enough then (for any realization of)..." it makes it sound as if we can choose $n$ which works uniformly for all $\omega$. It also doesn't hit the fact that it doesn't work for all $\omega$, but almost all $\omega$ but I'm not sure how to say this at the appropriate level for OP. $\endgroup$ – guy Feb 19 '12 at 15:05
  • $\begingroup$ I think you're using the word "parameter" too loosely. When you have a parametrized family of probability distributions, you have parameters. But $E(X)$ can exist regardless of anything about parametrized families. You don't have to think about more than one distribution---let alone a parametrized family of distributions---to speak of $E(X)$. $\endgroup$ – Michael Hardy Feb 19 '12 at 18:26
  • 1
    $\begingroup$ @MichaelHardy I disagree. Even if we're speaking of a single distribution, if $E(X)$ exists, I see no problem with calling it a parameter of the distribution of $X$. I suppose the term could be defined very narrowly, reserved for only those parameters which uniquely identify a distribution within a parametrized family, but that is too narrow, imho. I prefer to take parameter to mean "feature of a distribution." (For example, I would say the moments of a chi-square are parameters of the distribution, even though they are not the parameter we use to identify the distribution.) $\endgroup$ – William DeMeo Mar 24 '12 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.