# What is the motivation for the definition of the expected value?

I have a general question about expected values:

For a discrete random variable, $$E[X] = \sum_{i=1}^{\infty} x_{i}p_{i}$$ and $$E[X] = \int_{-\infty}^{\infty} xp(x) \ dx$$ for a continuous random variable $X$.

But what is the motivation for these definitions? Is it essentially defined because of the following: Suppose I perform an experiment a large number of times and record the results. I then take the average of the results. I want to find what this average approaches as the number of trials increases. Thus by trial and error I find that the definition of $E(X)$ works. This is assuming I am taking a frequentist view of probability. What does this all have to do with computing the area under some function?

-
It is more of a weighted average than a standard average, in that some values have more weight than others; the expected value is skewed (specifically, increases directly as a function of )both by the magnitude of a value (as in the case of standard averages), as well as by the probability of each value. But I don't see the frequentist part, since I don't see any mention of how the values $p_i$ were derived/ arrived-at in this case. –  gary Aug 9 '11 at 23:24
Yes. What you've said is true of both the discrete and continuous definitions, the latter of which is just the limit (say in the sense of Riemann sums) of the former. Is there more to say than that? –  Qiaochu Yuan Aug 9 '11 at 23:29
@Qiaochu Yuan: So the definition of $E(X)$ came through trial and error? And the weak law of large numbers came before the definition of expected value? –  Damien Aug 9 '11 at 23:34
@Damien: I'm not sure what you mean by "trial and error." Are you asking a historical question or a pedagogical one? –  Qiaochu Yuan Aug 9 '11 at 23:35
I've always felt the label "expected value" was misleading because of Petersburg-style paradoxes. To me, the "expected values," if they exist, would be the ones which globally maximize the probability density function. Then they could reasonably be expected, as the name implies, and might be somewhat more useful indicators than $\mu$ for practical decisions in the small run (as we all seem to have very much finite lives and resources). $E(X)$ is a limiting, asymptotic average, which carries different information about the distribution. –  anon Aug 20 '11 at 1:03

Let's go way back to the time of Fermat. You're a gambler. You play a game in which there are a bunch of events $i$ that randomly occur and give you $x_i$ units of money, and the gamemaster charges you $C$ units of money to play. You'd like to know whether he's ripping you off. So you play (or simulate) the game many, many times, and find that if you play the game $N$ times for large $N$, it turns out that approximately $p_i N$ of the time you get event $i$, where $p_i$ is some constant. So your total profit after playing the game $N$ times is approximately

$$\sum_i x_i p_i N - NE = N \left( \sum_i x_i p_i - C \right).$$

Now, if this number were positive, the gamemaster would be losing money in the long run, so would quickly go out of business. If this number were negative, you'd know you were getting ripped off, so you'd probably stop playing. The only way that the game is fair in the long run is if $C$ is exactly equal to the expected output of the game.

If the space of possible events is continuous rather than discrete, then the sum needs to be replaced by an integral. This just expresses the fact that Monte Carlo integration works.

-

Integrals can mean many things, area under a curve is just one. It can also be an antiderivative. If the measure space has total measure $1$, an integral can be the weighted mean of a function. In the case of expected value, the measure space, a probability space, has measure $1$: $$\sum_{i=1}^\infty\;\;p_i=1$$ or $$\int_{-\infty}^\infty\;p(x)\;dx=1$$ The "expected value" is really the expected average value of a random variable. The expected value may never actually occur. For example, the expected value of a standard six-sided die is $3.5$, but since the possible values are in $\{1,2,3,4,5,6\}$, $3.5$ will never occur, so it is never really "expected".

-

Professor Peter Whittle, Cambridge University, argued that average (expected value) was a more intuitive notion than probability, frequentist or not. For this reason he axiomatized expectation rather than probability. Kolmogorov axiomatized probability in 1933, and this is regarded as a milestone (millstone?) in the history of Probability Theory.

Whittle wrote a book in 1970 called Probability via Expectation. This book has been translated into Russian and other languages, and is now in its 4th Edition, 2000. Read the introductory chapter here: Probabilty via Expectation

Now, if you think that Peter Whittle's view of randomness is eccentric, then read what the great Rudolf Kalman, of Kalman Filter fame, has to say about Kolmogorov probability here:

www.scribd.com/doc/62685671/Kalman-Probability-in-the-Real-World

where he answers the question: Why is probability not a satisfactory way of looking at randomness?

Axiomatic Probability or Expectation? Take your pick -- randomly, of course.

-