3
$\begingroup$

The proof of linearity for expectation given random variables are independent is intuitive. What is the proof given there they are dependent?

Formally, $$ E(X+Y)=E(X)+E(Y)$$ where $X$ and $Y$ are dependent random variables.

The proof below assumes that $X$ and $Y$ belong to the sample space. That is, they map from the sample space to a real number line. Is that also a condition for linearity of expectation?

Proof: $$E(X+Y) =\sum\limits_{s}(X+Y)(s) P({s}) $$ $$E(X+Y) =\sum\limits_{s}(X(s)+Y(s)) P({s}) $$ $$E(X+Y) =\sum\limits_{s} X(s)P({s}+ \sum\limits_{s} Y(s)P({s}) $$ $$E(X+Y) =E(X)+E(Y)$$ Here $S$ is the sample space and $s$ is an event in the sample space.

Reference Lecture for proof.

Also, more reasoning for step 2 would be helpful. I don't understand it completely.

$\endgroup$
  • $\begingroup$ Second last line is missing a $P(s)$. $$\mathsf E(X+Y) = \sum_s X(s)P(s)+\sum_s Y(s)P(s)$$. Otherwise, it is okay. $\endgroup$ – Graham Kemp Jun 2 '16 at 23:56
  • $\begingroup$ @GrahamKemp thanks, changed. $\endgroup$ – Abhishek Bhatia Jun 2 '16 at 23:58
  • $\begingroup$ @Masacroso Are you saying the proof is invalid? Can you illustrate more please. $\endgroup$ – Abhishek Bhatia Jun 3 '16 at 0:10
  • 1
    $\begingroup$ Its justified by the definition of function operations. If $f, g$ are functions with the same domain, then so is $(f+g)$. $$(f+g)(x) := f(x)+g(x)$$ $\endgroup$ – Graham Kemp Jun 3 '16 at 0:25
  • $\begingroup$ @GrahamKemp Thanks, can you provide a reference for the statement if possible. $\endgroup$ – Abhishek Bhatia Jun 3 '16 at 20:42
13
$\begingroup$

The proof below assumes that $X$ and $Y$ belong to the sample space. That is, they map from the sample space to a real number line. Is that also a condition for linearity of expectation?

No.   It's the definition of a random variable.

Basically a random variable $X$ is a function that maps the sample space to the reals (or a subset there of, called the support).   $$X: \Omega \mapsto \Bbb R$$

If $X$ and $Y$ are both random variables of the same sample space, then so is their sum. $X+Y$.   (That is not defined if they are not of the same sample space.)  

$$ X:\Omega\mapsto\Bbb R~\wedge~ Y:\Omega\mapsto \Bbb R ~~\implies~~ X+Y:\Omega\mapsto\Bbb R\\\forall s\in\Omega,\quad(X+Y)(s) := X(s)+Y(s)$$

Linearity of Expectation then follows from its definition.

$\begin{align} \mathsf E(X+Y) =&~ \sum_{\omega\in\Omega} (X+Y)(\omega)~\mathsf P(\omega) \\[1ex] =&~ \sum_{\omega\in \Omega} X(\omega)~\mathsf P(\omega)+\sum_{\omega\in \Omega} Y(\omega)~\mathsf P(\omega) \\[1ex] =&~ \mathsf E(X)+\mathsf E(Y) \end{align}$

Of course, this is for discrete random variables.   For continuous random variables we use integration , but everything is analogous by no coincidence.

$\begin{align} \mathsf E(X+Y) =&~ \int_{\Omega} (X+Y)(\omega)~\mathsf P(\mathrm d \omega) \\[1ex] =&~ \int_{\Omega} X(\omega)~\mathsf P(\mathrm d \omega)+\int_{\Omega} Y(\omega)~\mathsf P(\mathrm d \omega) \\[1ex] =&~ \mathsf E(X)+\mathsf E(Y) \end{align}$

$\endgroup$
  • 1
    $\begingroup$ $Y$ might map from a different sample space $\Omega'$, but it only makes sense to talk about $X+Y$ if they are defined on the same sample space. So yes, you do have to assume $X$ and $Y$ map from the same sample space, the only reason being $X+Y$ is not well-defined otherwise. $\endgroup$ – kccu Jun 3 '16 at 0:01
  • $\begingroup$ @kccu ahha, of course! I didn't think hard enough. Thanks! $\endgroup$ – Abhishek Bhatia Jun 3 '16 at 0:07
  • $\begingroup$ what about the $\mathbb{P}$ bit? Do they have to be the same? So, let's say we have two Gaussian distributions, where should the formula for the Gaussians be put? Are they part of $\mathbb{P}$, or are they part of $X$ and $Y$? As far as I know, they should be part of $\mathbb{P}$, and then $X$ will be something like simply the Lebesgue measure, eg effectively simply something like $\omega$? $\endgroup$ – Hugh Perkins Mar 6 '17 at 10:03
  • $\begingroup$ Oh, I think I figured it out. If we have two independent gaussians, that's modeling two independent "things" happening, and therefore we need to add all possible outcomes, of each "thing" to the sample space. And we'll need to add all possible pairs of values. So basically the outcome space, instead of being eg $\mathbb{R}$ will become eg $\mathbb{R}^2$, with one axis for each of the things we want to measure, each of the two Gaussians. Then $X$ will be a projection of $\omega$ onto the first real axis, and $Y$ will be a projection onto the second axis. $\mathbb{P}$ will be the joint prob. $\endgroup$ – Hugh Perkins Mar 6 '17 at 11:56
  • $\begingroup$ Yes, for any continuous random variables with a definite joint probability density function, $f_{X,Y}(x,y)$, the above can be written: $$\def\P{\mathop{mathsf P}}\def\E{\mathop{\mathsf E}}\begin{align}\def\d{\mathop{\mathrm d}}\E(X+Y) ~&=~ \iint_{\Bbb R^2} (x+y)f_{X,Y}(x,y)\d x\d y\\[1ex] &=~ \iint_{\Bbb R^2} x\,f_{X,Y}(x,y)\d x\d y+\iint_{\Bbb R^2} y\,f_{X,Y}(x,y)\d x\d y\\[1ex] &=~ \int_\Bbb R x \,f_X(x)\d x+\int_\Bbb R y\,f_Y(y)\d y\\[1ex] &=~ \E(X)+\E(Y)\end{align}$$ $\endgroup$ – Graham Kemp Mar 6 '17 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.