16
$\begingroup$

The proof of linearity for expectation given random variables are independent is intuitive. What is the proof given there they are dependent?

Formally, $$ E(X+Y)=E(X)+E(Y)$$ where $X$ and $Y$ are dependent random variables.

The proof below assumes that $X$ and $Y$ belong to the sample space. That is, they map from the sample space to a real number line. Is that also a condition for linearity of expectation?

Proof: $$E\left(X+Y\right) =\sum\limits_{s}\left(X+Y\right)\left(s\right) P\left({s}\right) $$ $$E\left(X+Y\right) =\sum\limits_{s}\left(X\left(s\right)+Y\left(s\right)\right) P\left({s}\right) $$ $$E\left(X+Y\right) =\sum\limits_{s} X\left(s\right)P\left({s}\right) + \sum\limits_{s} Y\left(s\right)P\left({s}\right) $$ $$E\left(X+Y\right) =E\left(X\right)+E\left(Y\right)$$ Here $S$ is the sample space and $s$ is an event in the sample space.

Reference Lecture for proof.

Also, more reasoning for step 2 would be helpful. I don't understand it completely.

$\endgroup$
6
  • $\begingroup$ Second last line is missing a $P(s)$. $$\mathsf E(X+Y) = \sum_s X(s)P(s)+\sum_s Y(s)P(s)$$. Otherwise, it is okay. $\endgroup$ Commented Jun 2, 2016 at 23:56
  • $\begingroup$ @GrahamKemp thanks, changed. $\endgroup$ Commented Jun 2, 2016 at 23:58
  • $\begingroup$ @Masacroso Are you saying the proof is invalid? Can you illustrate more please. $\endgroup$ Commented Jun 3, 2016 at 0:10
  • 1
    $\begingroup$ Its justified by the definition of function operations. If $f, g$ are functions with the same domain, then so is $(f+g)$. $$(f+g)(x) := f(x)+g(x)$$ $\endgroup$ Commented Jun 3, 2016 at 0:25
  • $\begingroup$ @GrahamKemp Thanks, can you provide a reference for the statement if possible. $\endgroup$ Commented Jun 3, 2016 at 20:42

1 Answer 1

28
$\begingroup$

The proof below assumes that $X$ and $Y$ belong to the sample space. That is, they map from the sample space to a real number line. Is that also a condition for linearity of expectation?

No.   It's the definition of a random variable.

Basically any random variable $X$ is a function that maps the sample space to the reals (or a subset there of, called the support).   $$X: \Omega \mapsto \Bbb R$$

If $X$ and $Y$ are both random variables of the same sample space, then so is their sum. $X+Y$.   (That is not defined if they are not of the same sample space.)  

$$ X:\Omega\mapsto\Bbb R~\wedge~ Y:\Omega\mapsto \Bbb R ~~\implies~~ X+Y:\Omega\mapsto\Bbb R\\\forall s\in\Omega,\quad(X+Y)(s) := X(s)+Y(s)$$

Linearity of Expectation then follows from its definition.

$\begin{align} \mathsf E(X+Y) =&~ \sum_{\omega\in\Omega} (X+Y)(\omega)~\mathsf P(\omega) \\[1ex] =&~ \sum_{\omega\in \Omega} X(\omega)~\mathsf P(\omega)+\sum_{\omega\in \Omega} Y(\omega)~\mathsf P(\omega) \\[1ex] =&~ \mathsf E(X)+\mathsf E(Y) \end{align}$

Of course, this is for discrete random variables.   For continuous random variables we use integration , but everything is analogous by no coincidence.

$\begin{align} \mathsf E(X+Y) =&~ \int_{\Omega} (X+Y)(\omega)~\mathsf P(\mathrm d \omega) \\[1ex] =&~ \int_{\Omega} X(\omega)~\mathsf P(\mathrm d \omega)+\int_{\Omega} Y(\omega)~\mathsf P(\mathrm d \omega) \\[1ex] =&~ \mathsf E(X)+\mathsf E(Y) \end{align}$

$\endgroup$
11
  • 1
    $\begingroup$ $Y$ might map from a different sample space $\Omega'$, but it only makes sense to talk about $X+Y$ if they are defined on the same sample space. So yes, you do have to assume $X$ and $Y$ map from the same sample space, the only reason being $X+Y$ is not well-defined otherwise. $\endgroup$
    – kccu
    Commented Jun 3, 2016 at 0:01
  • $\begingroup$ @kccu ahha, of course! I didn't think hard enough. Thanks! $\endgroup$ Commented Jun 3, 2016 at 0:07
  • 4
    $\begingroup$ Yes, for any continuous random variables with a definite joint probability density function, $f_{X,Y}(x,y)$, the above can be written: $$\def\P{\mathop{mathsf P}}\def\E{\mathop{\mathsf E}}\begin{align}\def\d{\mathop{\mathrm d}}\E(X+Y) ~&=~ \iint_{\Bbb R^2} (x+y)f_{X,Y}(x,y)\d x\d y\\[1ex] &=~ \iint_{\Bbb R^2} x\,f_{X,Y}(x,y)\d x\d y+\iint_{\Bbb R^2} y\,f_{X,Y}(x,y)\d x\d y\\[1ex] &=~ \int_\Bbb R x \,f_X(x)\d x+\int_\Bbb R y\,f_Y(y)\d y\\[1ex] &=~ \E(X)+\E(Y)\end{align}$$ $\endgroup$ Commented Mar 6, 2017 at 14:39
  • 1
    $\begingroup$ @gwg If you prefer:$$\begin{align}\mathsf E(X+Y)&=\int_\Bbb R z~f_{X+Y}(z)\mathsf d z\\&=\iint_{\Bbb R^2} z~f_{X+Y,Y}(z,y)\mathsf d z\mathsf d y\\&= \iint_{\Bbb R^2} z~f_{X,Y}(z-y,y)\mathsf d z\mathsf d y\\&=\iint_{\Bbb R^2} (x+y)~f_{X,Y}(x,y)\mathsf d x\mathsf d y\\&\vdots\\&=\mathsf E(X)+\mathsf E(Y)\end{align}$$ $\endgroup$ Commented Mar 20, 2019 at 3:44
  • 5
    $\begingroup$ Law of Total Probability$$\int_\Bbb R f_{X,Y}(x,y)~\mathrm d x = f_Y(y)$$Also known as Marginalization. $\endgroup$ Commented Apr 20, 2019 at 5:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .