4
$\begingroup$

Please see the proof below regarding the linearity of expectation given two discrete random variables $X$ and $Y$. I'm not understanding how the first highlighted step moves to the next highlighted step. I've looked online and seen mention that this relates to the law of total probability... but after looking up some more information on the law of total probability I can't see how this law is applied here.

$$\begin{align*}E[X+Y]&=\sum_x\sum_y [(x+y)\cdot P(X=x, Y=y)]\\ &=\sum_x\sum_y [x\cdot P(X=x, Y=y)]+\sum_x\sum_y [y\cdot P(X=x, Y=y)]\\&=\bbox[yellow]{\sum_xx\sum_y P(X=x, Y=y)+\sum_xy\sum_y [P(X=x, Y=y)]}\\ &=\bbox[yellow]{\sum_x x \cdot P(X=x)+\sum_y y\cdot P(Y=y)}\\ &=E[X]+E[Y]\end{align*}$$

$\endgroup$
3
  • 1
    $\begingroup$ If you just think about it, the probability that $X$ takes the value $x$, written as $P(X = x)$, is equal to the sum of the probabilities of $X$ equaling $x$ and $Y$ equaling $y$ for all possible $y$. In other words, when $X = x$, each $y$ is a "case", i.e., a possible value that $Y$ can take. So to just get $P(X = x)$ you can sum up the probabilities of all the "cases" $\sum \limits_{y} P(X = x \text{ and } Y = y)$. $\endgroup$
    – layman
    Commented Nov 17, 2016 at 17:30
  • $\begingroup$ @user46944 But how do you guarantee the discrete random variable $Y$ has the same number of cases as $X$? For instance, if $X=\{1,2,3,4,5,6\}$ and $Y=\{3,4,5\}$? $\endgroup$ Commented Nov 17, 2016 at 22:29
  • 1
    $\begingroup$ You don't need any such guarantee. You just sum the value of X times the probability of the outcome for all possible outcomes. $$\sum\limits_{\omega\in\{1,2,3,4,5,6\}{\times}\{3,4,5\}} X(\omega)\,\Bbb P\{\omega\}$$ $\endgroup$ Commented Nov 17, 2016 at 22:49

2 Answers 2

4
$\begingroup$

The missing step is just that probability is additive for disjoint unions, and the support of a random variable partitions the sample space.   Also known as the Law of Total Probability.

$$\begin{align}\mathsf E(X+Y) &=\sum_x\sum_y (x+y)~\mathsf P(X=x\cap Y=y) \\[1ex] &= \sum_x\sum_y x~\mathsf P(X=x\cap Y=y)+\sum_x\sum_y y~\mathsf P(X=x\cap Y=y)\\[1ex]&=\sum_x x \sum_y\mathsf P(X=x\cap Y=y)+\sum_y y\sum_x\mathsf P(X=x\cap Y=y) \\[1ex] &= \sum_x x~\mathsf P\Bigl((X=x)\cap \bigcup_y (Y=y)\Bigr)+\sum_y y~\mathsf P\Bigl((Y=y)\cap\bigcup_x(X=x)\Bigr)\tag{$\bigstar$}\\[1ex] &= \sum_{x}x~\mathsf P(X=x)+\sum_{y}y~\mathsf P(Y=y)\\[1ex] &= \mathsf E(X)+\mathsf E(Y)\end{align}$$

PS: If you could write all probabilities of pairs of $(X,Y)$ values in a tabular grid and sum the values in the rows and columns into the margins, those values are the marginal probability masses.   This is where the name comes from.$$\mathsf P(X=x)=\sum_y \mathsf P(X=x, Y=y)$$

$\endgroup$
2
  • $\begingroup$ I'm still struggling just a bit with this... so just to be clear then $P(X=x)$ is the probability density function... so when I'm looking at this notation $\sum_y P(X=x, Y=y)$ when am I going through the entire range of all $x\in X$? For instance, considering $X=\{1,2,3,4,5,6\}$ and $Y=\{3,4,5\}$ would it work as follows: $P(X=1\vert Y=3)\cdot P(Y=3) + P(X=2\vert Y=3)\cdot P(Y=3) + P(X=3\vert Y=3)\cdot P(Y=3) + \cdots + P(X=6\vert Y=3)\cdot P(Y=3)$ and then starting the sum again at $P(X=1\vert Y=4)\cdot P(Y=4)$? $\endgroup$ Commented Nov 18, 2016 at 16:00
  • 1
    $\begingroup$ @ClownInTheMoon No; it is for any particular value of $x$, not all of them. The $y$ subscript of the sigma-operator indicates that is summing over all possible $y$ values (that lead to non-zero terms); in this case, the support of $Y$. $$\begin{align}\mathsf P(X{=}x)~&=~\sum_y \mathsf P(X{=}x, Y{=}y) \\[1ex]& =~\mathsf P(X{=}x, Y{=}3)+ \mathsf P(X{=}x, Y{=}4)+ \mathsf P(X{=}x, Y{=}5)\\[2ex]\mathsf P(X=2)~&=~\mathsf P(X{=}2, Y{=}3)+ \mathsf P(X{=}2, Y{=}4)+ \mathsf P(X{=}2, Y{=}5) \end{align}$$ $\sum_x\ldots$ would indicate summing over all possible $x$ values. $\endgroup$ Commented Nov 19, 2016 at 5:49
4
$\begingroup$

https://en.wikipedia.org/wiki/Law_of_total_probability

If you look at the first formula in the above link, we have in your case $P(X=x)=\sum_y P(X=x, Y=y)=\sum_y P(X=x\cap Y=y)$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .