# How to verify that the average is really a simple calculation?

There are many problems were the calculation of the actual probability is a hard one but evaluating the average (expected value) is easy.

Example
Shuffle two decks of cards and turn over each card of both decks simultaneously: How many times will you find the same card on each deck? Calculating the probability for at least one hit is quite tedious (via the inclusion-exclusion principle: $1-1/e$) but calculating the expected value of this happening is quite easy (via linearity of the expected value operator: 1).

It is interesting to note that the calculation of the average being easy is not dependent on the random variables being independent of each other. But what is important for this to work is that the marginal probabilities are not being changed by their potential dependence (so e.g. even if two throws of a die are not independent they can still be fair).

My question
How can you be sure that the marginal probabilities are not being changed by their dependence? This is: How can you differentiate situations were the calculation of the average is therefore easy and were even the average is hard to calculate (like the actual probabilities) because the marginal probabilities change?

-
Nice to see you picking up on the linearity of expectation :-). I'm not sure what you mean by "change" -- relative to which other situation? Could you give an example of a situation that you'd describe as "the marginal probabilities being changed by their dependence"? – joriki Sep 25 '11 at 10:24
@joriki: Perhaps this one is a little bit contrived but imagine two quantum dice where when one die is thrown the expected value of the other one changes from 3.5 to 4.5. Here they are not only dependent but the marginal probability changes in the experiment so you can't just calculate the overall expected value the easy way. I guess there are better real world examples out there... – vonjd Sep 25 '11 at 10:46
Hmmm. That example doesn't really help me understand what sort of answer you're looking for. In that example, the answer would just be: "Well, you explicitly said that performing the experiment physically changes the marginal probabilities for the other die, so why would you expect to be able to use the old marginal probabilities?" There's actually a physical change here, whereas your question seemed to suggest that you were thinking of some sort of mathematical "change" that I don't understand. Otherwise it would be a physics question about when experiments affect each other's outcomes. – joriki Sep 25 '11 at 10:53
Relating this question to this other question that presumably inspired this one: There's no physical change involved there. The probabilities of all matches are the same by symmetry, independent of the physical process you use to randomize them, and that equality was all I used in applying the linearity of expectation. – joriki Sep 25 '11 at 10:58
Yes, you are right! Thank you for clarifying. So, to turn it around: Can you always use the easy way to calculate the average value? Which would mean that there are no situations thinkable where the marginal probabilities change (independent anyway but even when dependent)? Can this be proved? – vonjd Sep 25 '11 at 11:01

OK, I'll try to answer this even though I'm still not sure I've properly understood what you mean by "change".

Linearity of expectation says that the expectation value of the sum of two random variables (defined with respect to the same probability space) is the sum of their expectation values (and analogously for linear combinations). This always holds and is simply a consequence of the definition of the expectation value as an integral with respect to the probability measure. (If you don't know this terminology, you can replace "probability space" by "set of events with probabilities assigned to them" and "integral with respect to the probability measure" by "sum/integral of values times probabilities").

Correctly applied, this holds also in the case of your quantum dice. Let's say we throw the dice one after the other, and the dice work such that the first die has a uniform distribution and the second die shows a number less than or equal to the one on the first die, with the same probability for all such numbers. Then the conditional expectation value for the second die, given that the first die showed $k$, is $(k+1)/2$, and the total expectation value for the second die is therefore

$$\sum_{k=1}^6\frac16\frac{k+1}2=\frac1{12}\left(\frac{6(6+1)}2+6\right)=2\frac14\;.$$

The expectation value for the first die is $3\frac12$, and you can easily verify that the expectation value for the sum is the sum of the expectation values, $5\frac34$.

Here we had to use conditional expectation values because the we had no independent way of knowing the expectation value of the second die; its probabilities are only given dependent on what the first die shows. That doesn't change the linearity of expectation, but it prevents us from using a shortcut to evaluate the expectation value without thinking about conditional probabilities.

In How to calculate the expected value when betting on which pairings will be selected, things were different in that we had an independent way of knowing the expectation value for each match. We could have figured out the expectation value for the second match by imagining some physical process that randomizes the matches, considering the draws in the process sequentially and adding up conditional expectation values for the second match depending on what was drawn in the first. But that wasn't necessary because whatever concrete physical process was chosen to randomize the matches, it was by definition chosen such that all teams and all matches are treated the same and the expectation value for all matches is the same. Thus, if we had unexpectedly found that the expectation value for the second match was different from the first, this would have been not a failure of the linearity of expectation, but a failure to choose an appropriate process to randomize the matches.

Given that the expectation values are all the same by definition, we don't have to go to all the hassle of figuring out conditional probabilities and expectation values; we can just multiply one of them by the number of matches.

Now you might ask: But what if we don't say they're the same by definition, but focus on some particular process of randomizing the matches and want to know if that particular process leads to equal expectation values for all matches, don't we then have to figure out the conditional probabilities? That's a question about symmetry, about how to tell whether a process treats all objects of a certain kind equally. In the present case, you presumably had in mind something like a process where balls bearing the names of the teams are successively drawn from an urn, and the question is then how to tell whether all the possible matches have the same probability of being drawn in the second draw "even though" two balls have already been removed. This is clearly the case, since the balls that were removed were themselves picked randomly with uniform distribution, so there's nothing there that could break the symmetry between the teams.

I hope I've made this slightly clearer; if it's still not clear, I suggest you take some time to come up with an example where what you call the "easy calculation" works even though you wouldn't expect it to, or vice versa.

-
I wonder if it's generally true that people with a physics background are those who use the term "expectation value", whereas probabilists and statisticians write "expected value". – Michael Hardy Sep 25 '11 at 13:47
@Michael: That may well be. I've never heard/read anything other than "expectation value" in physics courses/books, I think. – joriki Sep 25 '11 at 13:53
Thank you, joriki. The most enlightening part for me is: "[...] the question is then how to tell whether all the possible matches have the same probability of being drawn in the second draw "even though" two balls have already been removed. This is clearly the case, since the balls that were removed were themselves picked randomly with uniform distribution, so there's nothing there that could break the symmetry between the teams." – vonjd Sep 26 '11 at 5:19
@vonjd: Thanks for the feedback! – joriki Sep 26 '11 at 5:24