I am confused with summing two random variables. Suppose $X$ and $Y$ are two random variables denoting how much is gained from each two games. If two games are played together, we can gain $E[X] + E[Y]$ in total. I understand until here. However, in many textbooks, the equation $E[X+Y]=E[X]+E[Y]$ is given as an explanation to expectation of playing two games together. Explanation is more difficult than the result.

What does $X+Y$ and $E[X+Y]$ mean? We define $E[X]=\sum X_ip_i$. So, do we define $E[X+Y]=\sum (X_i+Y_i)p_i$ where $p_i$ is the same for both random variables.

What if $X$ denotes the equally likely outcomes $1, 2, 3$ and $Y$ denotes the equally likely outcomes $1, 2, 3, 4, 5$?


Good answers from @nrpeterson and @Lord_Farin.

For a practical demonstration consider an unbiased random number generator $i \in 6$, more commonly known as a dice.

Let $X$ be the result of throwing this dice once (an event), and $Y$ be the result of a subsequent throw (another independent event).

If you consider the sample space of each event they are the same and are:

$$\begin{array}{|c|c|c|c|c|c|c|} \hline \text{Result} & 1 & 2 & 3 & 4 & 5 & 6 \\ \hline \text{Probablity} & \frac{1}{6} & \frac{1}{6}& \frac{1}{6}& \frac{1}{6}& \frac{1}{6}& \frac{1}{6} \\ \hline \end{array}$$

As you say, $E[X]=\sum X_ip_i$, so it is easy to show that $E(X)=E(Y)=3.5$. Please note that the expected value is not actually a result that is achievable; this is not uncommon. For what this means you are moving into philosophy rather than mathematics as discussed here. Trivially $E(X)+E(Y)=7$

So what is $X+Y$? Clearly it is an integer $\in [2,12]$ but unlike $X$ and $Y$ not all outcomes are equally likely. Consider the joint distribution of $X+Y$:

$$\begin{array}{c|c|c|c|c|c|c|c|} & X & 1 & 2 & 3 & 4 & 5 & 6 \\ \hline Y \\ \hline 1 & & 2 & 3&4&5&6&7 \\ \hline 2&&3&4&5&6&7&8 \\ \hline 3&&4&5&6&7&8&9 \\ \hline 4&&5&6&7&8&9&10 \\ \hline 5&&6&7&8&9&10&11 \\ \hline 6&&7&8&9&10&11&12 \\ \hline \end{array}$$

There are 36 possibilities that give rise to the 11 possible outcomes, the most common being 7 with $p=\frac{1}{6}$ and the least common being 2 and 12 with $p=\frac{1}{36}$. The expected value $E[X+Y]=\sum(X_i+Y_i)p_i$ can be calculated and is 7, so $E[X+Y]=E[X]+E[Y]$.

  • 1
    $\begingroup$ With respect to "...this is only the case for independent events..."; doesn't the equation $E[X + Y] = E[X] + E[Y]$ follow from the linearity of expectation, regardless of whether $X$ and $Y$ are independent? $\endgroup$ – Emaad Ahmed Manzoor Mar 18 '15 at 9:09
  • $\begingroup$ Right you are! My bad $\endgroup$ – Dale M Mar 18 '15 at 10:40

When you are talking about two random variables, you need to think about their joint distribution - so, rather than talking about $P(X=i)$, you need to talk about $P(X=i\text{ and }Y=j)$, or, as we usually write it, $P(X=i,Y=j)$.

If it helps, think of it as randomly choosing a vector with two components - then calling the first component $X$ and the second component $Y$. You can think of $X$ and $Y$ as the separate outcomes of two experiments - which may or may not be related. So, $X$ could be how much you win in the first hand of poker, and $Y$ how much you win in the second. Then $X+Y$ is how much you won in the first two hands together.

With this in hand, for a function $f(x,y)$, we can define (for variables that take discrete values), $$ \mathbb{E}[f(X,Y)]=\sum_{x,y}f(x,y)\cdot P(X=x, Y=y). $$ So, in your particular case, $$ \mathbb{E}[X+Y]=\sum_{x,y}(x+y)P(X=x,Y=y)=\sum_{x,y}xP(X=x,Y=y)+\sum_{x,y}yP(X=x,Y=y). $$ Consider the first of these sums. Note $$ \sum_{x,y}xP(X=x,Y=y)=\sum_{x}x\sum_{y}P(X=x,Y=y). $$ The inner sum here is precisely $P(X=x)$: the event "$X=x$" is the same as the event "$X=x$ and $Y$ takes any value", whose probability is exactly this sum. So, $$ \sum_{x,y}xP(X=x,Y=y)=\sum_{x}x\sum_{y}P(X=x,Y=y)=\sum_{x}xP(X=x)=\mathbb{E}[X]. $$ Similarly, $$ \sum_{x,y}yP(X=x,Y=y)=\mathbb{E}[Y], $$ and combining these gives the formula $$ \mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y]. $$

  • $\begingroup$ Very nice proof and explanations, thank you. $\endgroup$ – jeremy radcliff Jul 3 '16 at 16:50
  • $\begingroup$ Consider the first of these sums. Note ∑x,yxP(X=x,Y=y)=∑xx∑yP(X=x,Y=y). How did you get that? Sorry if it is common knowledge. $\endgroup$ – q126y May 9 '18 at 10:37

All three $E$s are different. This is because they pertain to different event spaces: $E[X]$ for the event space of $X$, $E[Y]$ for the event space of $Y$, and $E[X+Y]$ for... Yes, for what?

As you noticed, it is not immediately obvious what the event space should be. Noting that this is your first question on Mathematics.SE, I will not bother you with excessive generalities (if you are interested, Wikipedia is not a bad place to start) and discuss only the case where $X$ and $Y$ are "independent", in that "information about $X$ does not influence the distribution of $Y$" (in the present example, this means just that what happens in game 1 does not influence the outcome of game 2 and vice versa).

So, let $X$ have event space $\Omega_X$ (so $\Omega_X$ is the set of all possible $X_i$; we follow established mathematical custom and denote elements of $\Omega_X$ by $\omega_X$), and $Y$ have event space $\Omega_Y$. In any case, to be able to discuss $X$ and $Y$, we should incorporate information about both $\Omega$s; this can be done best by considering the Cartesian product $\Omega_X \times \Omega_Y$ of tuples $(\omega_X,\omega_Y)$ with $\omega_X \in \Omega_X$ and $\omega_Y \in \Omega_Y$. It thus is a big collection of all possible combinations of outcomes of $X$ and $Y$.

Now we want to assign a probability distribution to $\Omega_X \times \Omega_Y$, that is, a mapping $\Pr:\Omega_X\times\Omega_Y \to \Bbb R_{\ge 0}$ such that $$\sum_{(\omega_X,\omega_Y)} \Pr(\omega_X,\omega_Y) = 1$$ This probability distribution (which is called the joint distribution for $X$ and $Y$) will govern the random variable $(X,Y)$ (consisting of the pair formed by the random variables $X$ and $Y$). To distinguish it from the distributions of $X$ and $Y$, we write $\Pr_{(X,Y)}$ from here on.

In our case, where $X$ and $Y$ are independent, since we don't expect $\omega_Y$ to be influenced in any way by $\omega_X$, and vice versa, we propose that $\Pr_{(X,Y)}(\omega_X,\omega_Y)$ is of the form $f(\omega_X)g(\omega_Y)$ with $f,g$ suitable functions. In this case, we find that:

$$1 = \sum_{(\omega_X,\omega_Y)} {\Pr}_{(X,Y)}(\omega_X,\omega_Y) = \sum_{\omega_X} \sum_{\omega_Y} f(\omega_X)g(\omega_Y) = \left(\sum_{\omega_X} f(\omega_X)\right)\left(\sum_{\omega_Y} g(\omega_Y)\right)$$

Now there are of course natural functions $f,g$ such that both of these sums are $1$. For example, we just take $f(\omega_X) = \Pr_X(\omega_X)$, where $\Pr_X$ is the function that takes $X_i$ to $p_i$ (which looks more natural as $\Pr_X(X_i) = p_i$).

So we define:

$${\Pr}_{(X,Y)}(\omega_X,\omega_Y) = {\Pr}_X(\omega_X)\Pr{}_Y(\omega_Y)$$

Using the notation $\Pr_X$ introduced above, we can now give a good definition of $E_X[f(X)]$ for any random variable $X$ and function $f$, namely by:

$$E_X[f(X)] := \sum_{\omega_X} f(\omega_X)\Pr{}_X(\omega_X)$$

For example, if $f$ is the identity function, then $E_X[f(X)]$ is not but the familiar $E[X]$ (which you can check by translating all the symbols to the original notation).

Now with this notation, we can make sense of $E[X+Y]$ as $E_{(X,Y)}[X+Y]$, where we use the addition function $+$ which takes $(\omega_X,\omega_Y)$ to $\omega_X+\omega_Y$ (this is just addition of real numbers).

As a first exercise, you can try to prove (!) that $E_{(X,Y)}[X+Y] = E_X[X]+E_Y[Y]$.

As you can see, there is quite a bit of implicit machinery behind the notation of probability theory. Although you may find the above daunting, it is my personal experience that one cannot fully grasp what's going on (beyond the level of "applying the trick", which is a bad way of teaching because it does not yield any insight). With understanding the basics, you can then progress to more difficult results or other applications of statistics without always having to look up the formula to calculate anything. :)

  • $\begingroup$ Just to mention that the construction presented in this post is by no means necessary and would seem quite exotic to most probabilists. Actually, when computing E(X+Y), both random variables X and Y, independent or not, are defined from the start on a common probability space Omega. To sum up, the opening salvo "All three Es are different. This is because they pertain to different event spaces" is simply wrong. $\endgroup$ – Did Mar 18 '15 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.