I'm studying probability for preparing myself for machine learning. I came across this notion of Expectation of (random variable) or (function of random variable) like $E[X]$ or $E[g(X)]$. Can anybody explain me the intuitive idea of the above notion (taking into consideration it's implications in machine learning, may be)


Expected value means the average value of the random variable. It is a weighted average of the values of the function, weighted by the probabilities of taking those values.

Expected value is a linear map from some random variables to numbers. This map throws away all of the information about a random variable except one number. This number is the center of mass of the distribution.

In case the random variable is a statistic you can measure from a sample, you can imagine repeating independent samples many times. In that case, the Strong Law of Large Numbers (a theorem) says that the average of the results will be close to the expected value in that the limit of the averages will converge to the expected value. This does not mean that you need a statistic to come from a repeated experiment for the expected value to make sense. Expected value makes sense even if you can only observe a statistic once.

If your text doesn't spend a lot of time on the central and fundamental idea of expected value, it could be that the author assumes you have already read another text which has done this, and so you might look for a more elementary text.

| cite | improve this answer | |
  • $\begingroup$ Thanks for the answer. So, in case of NORMAL mean we just add the elements (considering 100% probability of each of them, which's rarely the case in the real world) & divide them...etc, but in case of EXPECTATION, the probability of individual elements (contributing to average) counts, so that the the average makes a real-world (full of uncertain info.) sense. Is that so? Correct me if I'm wrong somewhere. $\endgroup$ – amitlan Apr 3 '11 at 5:02
  • $\begingroup$ Mean and expected value are interchangeable. Both are weighted averages. $\endgroup$ – Douglas Zare Apr 11 '11 at 10:18

Think of you operate a gaming machine in a casino and it has $0$ $10$ $20$ $30$ $40$ $50$ usd prizes with each prize having probability of $p_1,p_2,\ldots p_6$,

If X is random variable defining amount of the prize, each time a game playing person wins, then say $E[X] = 5$ means that, on average lets say $1000$ people played a game, casino would pay $5000$ from its cash.

And again assume that for playing a game each time you buy a ticket which is $5$ USD.

Now $X$ is the prize amount you win, $5$ USD ticket fee, now casino gives $X$ USD to you but you give $5$ USD to casino. Now casino loses $(5-X)$ amount of money each time if $x=0$ it wins $5$ USD if $x=50$ it loses $45$ USD. I as an owner of casino want to calcculate long term profit or loss I use $E[f(X)] = E[5-x]$, now if I already know $E[X]$ in some way it its easy to calculate $E[f(x)]$

Brief note: I'm not an expert in probability so some of my intuitions might be wrong anyone please correct if so.

| cite | improve this answer | |

In calculating the average value of a random variable the underlying assumption is that all possible outcomes of the random variable are equally probable.

Let's say that random variable $X$ has $3$ possible outcomes, $x_1,x_2,x_3,$ and all three are equally probable.

The average value of variable $X$ is : $(x_1+x_2+x_3)/3$ or $0.333(x_1) + 0.333(x_2) + 0.333(x_3)$

Also the probability of each outcome is 0.3333. So the expectation is : $$ E(X) = 0.333(x_1+x_2+x_3) = (x_1 +x_2 + x_3)/3 $$

In the case where the outcomes were not equally probable and occurred with a probability of $p_1,p_2\text{ and } p_3$ respectively, then $$ E(X) = p_1\cdot x_1 + p_2\cdot x_2 + p_3\cdot x_3 $$ for the discrete random variable $X$.

Generalizing, we could write $$ E(X) = \sum p_i\,x_i $$ where the sum is over all outcomes.

| cite | improve this answer | |
  • 2
    $\begingroup$ Welcome to the site! I LaTeX-ified your answer. Please feel free to roll back my edits if you think I messed up your exposition. $\endgroup$ – Rick Decker Jun 20 '12 at 1:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.