0
$\begingroup$

I am having hard time understanding Linearity of Expectations. Here is my understanding.

$X$ is a random variable defined on Sample Space $S$ as $Pr\{X=x\}= \sum_{s \in S:X(s)=x}Pr\{s\}$

Expectation of Random variable is weighted average defined as $E[X]= \sum_{x}x.Pr\{X=x\}$.

Also, the Linearity of Expectation says, if $X$ is a random variable which is sum of other random variables such as $X_1,X_2,X_3,...X_n$ i.e., $X=X_1+X_2+X_3+.....+X_n$, then $$E[X]=E[X_1]+E[X_2]+E[X_3+....+E[X_n]=\sum_{i=1}^nE[X_i]$$.

Let's image we are doing a $n$-Bernoulli trial in which success probability is $p$ and failure probability is $q$ and we defined a random variable $X=$ Number of Successes where $X$ could take values $\{0,1,2,3....,n\}$.This gives $$E[X]=0.Pr\{X=0\}+1.Pr\{X=1\}+2.Pr\{X=2\}+3.Pr\{X=3\}+....+n.Pr\{X=n\}$$ I could see that $E[X]=np$ by the definition of binomial distribution formed by Bernoulli trials and applying some manipulations on it.

In the book I am reading, it is said that, instead of doing complex algebra to find the expectation, we can express X as sum of smaller random variables and apply Linearity of Expectation.Let $X_i$ is the random variable which describes number of successes in $i^{th}$ trial. It is said that $X=X_1+X_2+X_3+....+X_n$ (I don't understand why this is true).

Also, in order to find the $E[X_i]$ we are using Indicator Random Variables in which $Pr\{X=x_i\}=1.p+0.q=p$, that means $X$ can take values $0,1$.

I don't understand why a random variable which describes number of successes in all the trials is sum of successes in each trial.

Also, when we are computing original Expectation which is $E[X]$ we are working on sample space which has $2^n$ (domain of $X$) sample points, cause each trial has two possible outcomes and we have $n$ trials.

To calculate Expectation of individual random variables why did we shrink our sample space to two? I know it is due to the fact that, we are working on per trail basis but my expectation is, all the individual random variables also should be defined on original number of trails, not per trail.

If I do the math, I could get the correct value but what I lack here is intuition

$\endgroup$
1
  • $\begingroup$ The random variables $X,X_1,\dots,X_n$ are all defined on the same probability space (so there is no shrinking of our sample space, as you suggest). If that would not be the case then the expression $X=X_1+\cdots+X_n$ would have no sense at all. $\endgroup$
    – drhab
    Jul 10, 2018 at 7:55

1 Answer 1

0
$\begingroup$

For simplicity let $n=2$ and throw a coin twice.

We speak of a success if head turns up and $X$ is by definition the number of successes, so takes values in $\{0,1,2\}$.

Further define $X_1$ by stating that $X_1$ takes value $1$ if the first throw gives heads, and takes value $0$ otherwise.

Likewise define $X_2$ by stating that $X_2$ takes value $1$ if the second throw gives heads, and takes value $0$ otherwise.

Can you grasp that $X=X_1+X_2$?

If not directly then look at all $4$ different outcomes we can get: $$\begin{array}{ccccc} & HH & HT & TH & TT\\ X & 2 & 1 & 1 & 0\\ X_{1} & 1 & 1 & 0 & 0\\ X_{2} & 1 & 0 & 1 & 0\\ X_{1}+X_{2} & 2 & 1 & 1 & 0 \end{array}$$


For finding e.g. the expectation of a Bernoulli random variable $B$ we only have to know the value of $P(B=1)$. The looks of the underlying sample space are irrelevant.


edit

To calculate Expectation of individual random variables why did we shrink our sample space to two?

We do not!

Let us build a model for the situation that you describe. As sample space we can use the set $\Omega:=\{0,1\}^n$. Every subset of $\Omega$ is an event and must be equipped in a suitable way with a probability. If the probability on success is $p$ then tuple $\omega=(\omega_1,\dots,\omega_n)\in\{0,1\}^n$ induces an event $\{\omega\}\subseteq\Omega$ and it is equipped with probability $p^{\sum_{i=1}^n\omega_i}(1-p)^{n-\sum_{i=1}^n\omega_i}$ to occur. So for a subset $B\subseteq\{0,1\}^n$ we have $$P(B)=\sum_{\omega\in B}p^{\sum_{i=1}^n\omega_i}(1-p)^{n-\sum_{i=1}^n\omega_i}$$

Any function $Y:\Omega\to\mathbb R$ is a random variable.

Examples are:

  • function $X$ prescribed by $\omega\mapsto\sum_{i=1}^n\omega_i$
  • for $i=1,\dots, n$ function $X_i$ prescribed by $\omega\mapsto\omega_i$

Now observe that for every $\omega\in\Omega$ we have:$$X(\omega)=X_1(\omega)+\cdots+X_n(\omega)$$ leading to:$$\mathbb EX=\int X(\omega)P(d\omega)=\int X_1(\omega)+\cdots+X_n(\omega)P(d\omega)=\sum_{i=1}^n\int X_i(\omega)P(d\omega)=\sum_{i=1}^n\mathbb EX_i$$

Function $X$ has binomial distribution with parameters $n$ and $p$.

The functions $X_i$ are iid and have Bernoulli distribution with parameter $p$.

$\endgroup$
7
  • $\begingroup$ I can see it with examples. But, what I lack here is intuition. I tried it with3-Bernoulli trials. What I could not understand is how is it working when we represent one random variable on n-trials as sum of Indicator Random Variables on individual trial $\endgroup$
    – Arun Rahul
    Jul 9, 2018 at 18:21
  • $\begingroup$ I have edited my answer. $\endgroup$
    – drhab
    Jul 9, 2018 at 20:24
  • $\begingroup$ I got till summation part. Could you please explain the integration part differently? I am not well versed with integration stuff. $\endgroup$
    – Arun Rahul
    Jul 11, 2018 at 9:10
  • $\begingroup$ Integration comes in to define $\mathbb EX$. Familiar with that? Integration is linear: $\int f+gd\mu=\int fd\mu+\int gd\mu$. Familiar with that? That is all you need to know here. If not familiar then you should look it up in the math script/book that you have at your disposal. $\endgroup$
    – drhab
    Jul 11, 2018 at 9:44
  • $\begingroup$ I think I get it now. With the Random Variables you declared above, it is clear that summation of individual random variables $X_i$ equals to $X$. So, it is wrong that sample space is shrinked to 2 (S,F), we are still calculating the Expectation of individual random variables with sample space $2^n$. So, $E(X_i)=1 \times (1.1.1.......1.p) + 0 \times (1.1.1.......1.q) = p$ where $p$ and $q$ in the multiplication signifies the success and failure of trial $i$. Did I get that correct? $\endgroup$
    – Arun Rahul
    Jul 13, 2018 at 6:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .