# Proof of linearity for expectation given random variables are dependent

The proof of linearity for expectation given random variables are independent is intuitive. What is the proof given there they are dependent?

Formally, $$E(X+Y)=E(X)+E(Y)$$ where $$X$$ and $$Y$$ are dependent random variables.

The proof below assumes that $$X$$ and $$Y$$ belong to the sample space. That is, they map from the sample space to a real number line. Is that also a condition for linearity of expectation?

Proof: $$E\left(X+Y\right) =\sum\limits_{s}\left(X+Y\right)\left(s\right) P\left({s}\right)$$ $$E\left(X+Y\right) =\sum\limits_{s}\left(X\left(s\right)+Y\left(s\right)\right) P\left({s}\right)$$ $$E\left(X+Y\right) =\sum\limits_{s} X\left(s\right)P\left({s}\right) + \sum\limits_{s} Y\left(s\right)P\left({s}\right)$$ $$E\left(X+Y\right) =E\left(X\right)+E\left(Y\right)$$ Here $$S$$ is the sample space and $$s$$ is an event in the sample space.

Reference Lecture for proof.

Also, more reasoning for step 2 would be helpful. I don't understand it completely.

• Second last line is missing a $P(s)$. $$\mathsf E(X+Y) = \sum_s X(s)P(s)+\sum_s Y(s)P(s)$$. Otherwise, it is okay. Commented Jun 2, 2016 at 23:56
• @GrahamKemp thanks, changed. Commented Jun 2, 2016 at 23:58
• @Masacroso Are you saying the proof is invalid? Can you illustrate more please. Commented Jun 3, 2016 at 0:10
• Its justified by the definition of function operations. If $f, g$ are functions with the same domain, then so is $(f+g)$. $$(f+g)(x) := f(x)+g(x)$$ Commented Jun 3, 2016 at 0:25
• @GrahamKemp Thanks, can you provide a reference for the statement if possible. Commented Jun 3, 2016 at 20:42

The proof below assumes that $$X$$ and $$Y$$ belong to the sample space. That is, they map from the sample space to a real number line. Is that also a condition for linearity of expectation?

No.   It's the definition of a random variable.

Basically any random variable $$X$$ is a function that maps the sample space to the reals (or a subset there of, called the support).   $$X: \Omega \mapsto \Bbb R$$

If $$X$$ and $$Y$$ are both random variables of the same sample space, then so is their sum. $$X+Y$$.   (That is not defined if they are not of the same sample space.)

$$X:\Omega\mapsto\Bbb R~\wedge~ Y:\Omega\mapsto \Bbb R ~~\implies~~ X+Y:\Omega\mapsto\Bbb R\\\forall s\in\Omega,\quad(X+Y)(s) := X(s)+Y(s)$$

Linearity of Expectation then follows from its definition.

\begin{align} \mathsf E(X+Y) =&~ \sum_{\omega\in\Omega} (X+Y)(\omega)~\mathsf P(\omega) \\[1ex] =&~ \sum_{\omega\in \Omega} X(\omega)~\mathsf P(\omega)+\sum_{\omega\in \Omega} Y(\omega)~\mathsf P(\omega) \\[1ex] =&~ \mathsf E(X)+\mathsf E(Y) \end{align}

Of course, this is for discrete random variables.   For continuous random variables we use integration , but everything is analogous by no coincidence.

\begin{align} \mathsf E(X+Y) =&~ \int_{\Omega} (X+Y)(\omega)~\mathsf P(\mathrm d \omega) \\[1ex] =&~ \int_{\Omega} X(\omega)~\mathsf P(\mathrm d \omega)+\int_{\Omega} Y(\omega)~\mathsf P(\mathrm d \omega) \\[1ex] =&~ \mathsf E(X)+\mathsf E(Y) \end{align}

• $Y$ might map from a different sample space $\Omega'$, but it only makes sense to talk about $X+Y$ if they are defined on the same sample space. So yes, you do have to assume $X$ and $Y$ map from the same sample space, the only reason being $X+Y$ is not well-defined otherwise.
– kccu
Commented Jun 3, 2016 at 0:01
• @kccu ahha, of course! I didn't think hard enough. Thanks! Commented Jun 3, 2016 at 0:07
• Yes, for any continuous random variables with a definite joint probability density function, $f_{X,Y}(x,y)$, the above can be written: \def\P{\mathop{mathsf P}}\def\E{\mathop{\mathsf E}}\begin{align}\def\d{\mathop{\mathrm d}}\E(X+Y) ~&=~ \iint_{\Bbb R^2} (x+y)f_{X,Y}(x,y)\d x\d y\\[1ex] &=~ \iint_{\Bbb R^2} x\,f_{X,Y}(x,y)\d x\d y+\iint_{\Bbb R^2} y\,f_{X,Y}(x,y)\d x\d y\\[1ex] &=~ \int_\Bbb R x \,f_X(x)\d x+\int_\Bbb R y\,f_Y(y)\d y\\[1ex] &=~ \E(X)+\E(Y)\end{align} Commented Mar 6, 2017 at 14:39
• @gwg If you prefer:\begin{align}\mathsf E(X+Y)&=\int_\Bbb R z~f_{X+Y}(z)\mathsf d z\\&=\iint_{\Bbb R^2} z~f_{X+Y,Y}(z,y)\mathsf d z\mathsf d y\\&= \iint_{\Bbb R^2} z~f_{X,Y}(z-y,y)\mathsf d z\mathsf d y\\&=\iint_{\Bbb R^2} (x+y)~f_{X,Y}(x,y)\mathsf d x\mathsf d y\\&\vdots\\&=\mathsf E(X)+\mathsf E(Y)\end{align} Commented Mar 20, 2019 at 3:44
• Law of Total Probability$$\int_\Bbb R f_{X,Y}(x,y)~\mathrm d x = f_Y(y)$$Also known as Marginalization. Commented Apr 20, 2019 at 5:06