# Find the value of $\mathbb{E}(X_1+X_2+\ldots+X_N)$ of i.i.d random variables $X_i$s.

Let $X_1,X_2,X_3 ,\ldots$ be a sequence of i.i.d. random variables with mean $1$. If $N$ is a geometric random variable with the probability mass function $\mathbb{P}(N=k)=\dfrac{1}{2^k}$;
$k=1,2,3,\ldots$ and it is independent of the $X_i$'s, then $\mathbb{E}(X_1+X_2+\ldots+X_N)=$?

My work:

Since mean of $X_i$s are $1$, so $\frac{X_1+X_2+\ldots+X_N}{N}=1$. Hence $\mathbb{E}(X_1+X_2+\ldots+X_N)=\mathbb{E}(N)=\sum_{k=1}^{\infty}k\mathbb{P}(N=k) = \sum_{k=1}^\infty \dfrac{k}{2^k}=2$.

Is my solution correct? Is this the right process to do it? Help please. Thanks.

• The conditional expectation argument is fundamentally correct, but perhaps has not been made explicit enough. And there is a quite wrong assertion in the middle, namely $\frac{X_1+\cdots+X_N}{N}=1$. Jul 21, 2016 at 7:06
• I also thought about that middle part, may be incorrect, but I don't know how to use the fact that "mean is $1$", so I did that. @AndréNicolas. So how can I solve it correctly? Jul 21, 2016 at 7:08
• @HarryPotter Sorry to mention it but the accepted answer below does not solve the question.
– Did
Jul 23, 2016 at 15:42
• Thank you @Did for your pointer to a crucial step. Jul 24, 2016 at 14:22

Only a way to solve it correctly. In the answers of Ant and André it is explained what you did wrong.

If $S_N:=X_1+\cdots+X_N$ then: $$\mathbb ES_N=\sum_{k=1}^{\infty}\mathbb E(S_N\mid N=k)\Pr(N=k)$$

Observe that $\mathbb E(S_N\mid N=k)=\mathbb ES_k=\mathbb EX_1+\cdots+\mathbb EX_k=k$.

I leave the rest to you.

Edit

To explain why $\mathbb E(S_N\mid N=k)=\mathbb ES_k$, observe that $\left\{ S_{N}\in A\wedge N=k\right\} =\left\{ S_{k}\in A\wedge N=k\right\}$ for any measurable $A$, leading to:$$\Pr\left(S_{N}\in A|N=k\right)=\frac{\Pr\left(S_{N}\in A\wedge N=k\right)}{\Pr\left(N=k\right)}=\frac{\Pr\left(S_{k}\in A\wedge N=k\right)}{\Pr\left(N=k\right)}=\Pr\left(S_{k}\in A\right)$$

The last equation is a consequence of the fact that $N$ and $S_k$ are independent. So the distribution of $S_N$ under condition $N=k$ is the same as the distribution of $S_k$.

No, your process does not make any sense. Even if the mean of $X_k$ is $1$, this does not mean that $$\frac {X_1 + \dots + X_N}{N} = 1$$

for some $N$.

This is a huge misunderstanding; I'll try to briefly explain why is it so.

Random variables are not numbers

When talking about a random variable, you're talking about a function. When you write $X_1$, what you mean is $X_1(\omega)$; that is, it takes a different value for different $\omega$. As such, you cannot expect $X_1(\omega) + \dots + X_N(\omega) = N(\omega)$ to hold! The left hand side depends on the particular $\omega$, as does the right hand side. Since they are independent, this equation cannot hold for any $N$ (random variable or not)

Relationship between mean and expectation

Your confusion probably stems from the fact that if $x_i$ are numbers, then we call their "mean" (or more properly average) the number $\frac 1n\sum_{i=1}^n x_i$. Clearly you can do the same with random variables (and the result will again be a random variable, not a number) to get to

$$\frac 1n\sum_{i=1}^n X_i(\omega)$$

(note that in this context I am assuming $n$ constant, different than $N(\omega)$ random variable)

A priori, this value has no relationship whatsover with the abstract integral $E[X_1]$ (which is what we call the mean of the random variable). They are completely different things defined in completely different ways! For instance, $E[X_1]$ only depends on $X_1$ while the other is an average of many random variables.

One discovers, after some work, that if the random variables $X_i$ are independent and indentically distributed with mean $\mu \in \mathbb R$, then $$\frac 1n \sum_{i=1}^n X_i \to \mu$$

(the limit is taken a.s. and in $L^1$).

So you if you take a lot of iid random variables and you take their average, the result is still a random variable, but it's "very close" to a constant, and that constant is precisely the mean $\mu$. As $n \to \infty$, it will be "equal" to the constant $\mu$. This is called the Law of Large Numbers and it's the justification as of why we call $E[X_1]$ "mean" in the first place, but you should remember how things are defined and not get confused by words

How to solve the problem

Now, if instead of $N(\omega)$ we simply had $n$, the problem was easy; use linearity of expectation to find that

$$E(X_1 + \dots + X_n) = E(X_1) + \dots + E(X_n) = n$$

If $N(\omega)$ is a random variable, clearly you cannot do the same; you need to use the tower property of conditional expectation to write that

$$E(X_1 + \dots + X_N) = E(E(X_1 + \dots + X_n \mid N = n) ) = E(N) = \sum_{k=1}^\infty \frac k{2^k} = 2$$

• Thanks for the explanation, I was so dumb writing such things. Jul 21, 2016 at 7:25
• @HarryPotter Don't be discouraged, the important thing is to clear the confusion up :-)
– Ant
Jul 21, 2016 at 7:27
• Yes. You have helped me to clear these doubts now. Main thing is I learned some new things today. Thanks again. Jul 21, 2016 at 7:29
• Re the "How to solve this problem" part, note that, oddly enough, the independence hypothesis (which is crucial for the result to hold) is not even mentioned (a sure sign that something goes wrong). The offending step is of course when one states with no justification that $$E(X_1 + \dots + X_N) = E(E(X_1 + \dots + X_n \mid N = n) ) = E(N).$$ A sum over $n$ might be missing there, but even then, one should explain why, in the context of the question, for every $n$, $$E(X_1 + \dots + X_n \mid N = n)=n.$$
– Did
Jul 23, 2016 at 15:40

The calculation is incompletely justified, and one should not write $\frac{X_1+\cdots+X_N}{N}=1$.

Observe that $E(X_1+\cdots+X_N\mid N=k)=k$. This is because the $X_i$ and $N$ are independent, so for fixed $k$, we are just looking at $X_1+\cdots +X_k$, and by the linearity of expectation, the expectation of the sum is $(k)(1)$.

By the Law of Total Expectation, $$E(X_1+\cdots+X_N)=\sum_1^\infty E(X_1+\cdots+X_N\mid N=k)\Pr(N=k).$$ Thus we conclude that $$E(X_1+\cdots+X_N)=\sum_1^\infty k2^{-k},$$ and you calculated this sum correctly.

By Wald's identity (basic version),

$$E\left[ \sum_{i=1}^{N} X_i\right] = E[N] E[X_1]$$

where

$$E[N] = \sum_{k=1}^{\infty} \frac{k}{2^k}$$

$$E[X_1] = 1$$