Variance over IID a random number of times Let $X_1, X_2,\dotsc$ be independent and identically distributed with mean $E[X]$ and variance $VAR[X]$. Let $N$ be a non-negative integer-valued random variable independent of the $X_i$'s. Show
$$
VAR\left[ \sum_{i=1}^N X_i \right] = E[N]VAR[X]+(E[X])^2VAR[N]
$$
I've tried expanding this in a number of different ways but I can't quite seem to get it to work out. I don't really understand how to condition on a random variable like this. Any help would be greatly appreciated.
 A: The law of total variance says
$$
\operatorname{var}(Y) = \operatorname{E}(\operatorname{var}(Y\mid X)) + \operatorname{var}(\operatorname{E}(Y\mid X)).
$$
So
$$
\begin{align}
\operatorname{var}\left(\sum_{i=1}^N X_i\right) & = \operatorname{E} \left(\operatorname{var}\left(\sum_{i=1}^N X_i \mid N\right)\right) + \operatorname{var}\left(\operatorname{E} \left(\sum_{i=1}^N X_i \mid N\right)\right) \\  \\  \\
& = \operatorname{E}(N\operatorname{var}(X)) + \operatorname{var}(N\operatorname{E}(X)) \\  \\
& = \operatorname{var}(X)\operatorname{E}(N) + (\operatorname{E}(X))^2 \operatorname{var}(N).
\end{align}
$$
A: Performing repeated integration yields
$$
\begin{align}
\operatorname{E}[X]
&=\operatorname{E}_Y[\operatorname{E}_X[X\mid Y]]\tag{1}
\end{align}
$$
Applying $(1)$ to $X^2$ and using the fact that $\operatorname{Var}[X]=\operatorname{E}\left[X^2\right]-\operatorname{E}[X]^2$, we get
$$
\begin{align}
\operatorname{E}\left[X^2\right]
&=\operatorname{E}_Y\left[\operatorname{E}_X\left[X^2\mid Y\right]\right]\\
&=\operatorname{E}_Y\left[\operatorname{Var}_X[X\mid Y]\right] +\operatorname{E}_Y\left[\operatorname{E}_X[X\mid Y]^2\right]\tag{2}
\end{align}
$$
Applying $(1)$ and $(2)$, we get
$$
\begin{align}
\operatorname{Var}[X]
&=\operatorname{E}\left[X^2\right]-\operatorname{E}[X]^2\\
&=\operatorname{E}_Y\left[\operatorname{Var}_X[X\mid Y]\right] + \operatorname{E}_Y\left[\operatorname{E}_X[X\mid Y]^2\right]-\operatorname{E}_Y[\operatorname{E}_X[X|Y]]^2\\
&=\operatorname{E}_Y\left[\operatorname{Var}_X[X\mid Y] \right] +\operatorname{Var}_Y[\operatorname{E}_X[X\mid Y]]\tag{3}
\end{align}
$$
Now apply $(3)$ to the problem:
$$
\begin{align}
\operatorname{Var}\left[\sum_{i=1}^NX_i\right]
&=\operatorname{E}_N\left[\left.\operatorname{Var}_X\left[\sum_{i=1}^NX_i\right]\right|N\right]+\operatorname{Var}_N\left[\left.\operatorname{E}_X\left[\sum_{i=1}^N X_i\right]\right|N\right]\\
&=\operatorname{E}_N[N\operatorname{Var}[X]]+\operatorname{Var}_N[N\operatorname{E}[X]]\\
&=\operatorname{E}[N]\operatorname{Var}[X]+\operatorname{Var}[N]\operatorname{E}[X]^2\tag{4}
\end{align}
$$
A: Another way to do this: let $Y_i = 1$ if $N \ge i$, $0$ otherwise.  Then your sum is $$S = \sum_{i=1}^N X_i = \sum_{i=1}^\infty Y_i X_i$$ (I won't worry about convergence of infinite sums: if you wish you can use a truncated version of $N$ and then take limits). So $$\text{var}(S) = \sum_{i=1}^\infty \text{var}(Y_i X_i) + 2 \sum_{i=1}^\infty \sum_{j=1}^{i-1} \text{cov}(Y_i X_i, Y_j X_j)$$
Now $\text{var}(Y_i X_i) = E[Y_i^2 X_i^2] - E[Y_i X_i]^2 =  E[Y_i] \text{var}(X) + \text{var}(Y_i) E[X]^2$, while
for $j < i$, $\text{cov}(Y_i X_i, Y_j X_j) = E[Y_i Y_j X_i X_j] - E[Y_i X_i] E[Y_j X_j] 
= E[Y_i] (1 - E[Y_j]) E[X]^2$, so that
$$ \text{var}(S) = \sum_{i=1}^\infty E[Y_i] \text{var}(X) + \sum_{i=1}^\infty \text{var}(Y_i) E[X]^2 + 2 \sum_{i=1}^\infty \sum_{j=1}^{i-1} E[Y_i](1 - E[Y_j]) E[X]^2$$
Now doing the same calculation with $X_i$ replaced by 1 (since  $N = \sum_{i=1}^\infty Y_i$),
$$ \text{var}(N) = \sum_{i=1}^\infty \text{var}(Y_i) + 2 \sum_{i=1}^\infty \sum_{j=1}^{i-1} E[Y_i](1 - E[Y_j])$$
so that
$$ \text{var}(S) = \sum_{i=1}^\infty E[Y_i] \text{var}(X) + \text{var}(N) E[X]^2
= E[N] \text{var}(X) + \text{var}(N) E[X]^2 $$
A: Let $Y = \sum_{i=1}^N X_i$. Notice, that the characteristic function of $Y$ can be expressed as composition of characteristic functions of $X$ $\phi(t)$ and the probability generating function of $N$, $g(s)$:
$$
 \psi(t) = \mathbb{E}( \exp( i Y t) ) = \mathbb{E}\left( \mathbb{E}( \exp( i Y t) \vert N) \right) =
    \mathbb{E}\left( \phi(t)^N \right) = \sum_{k=0}^\infty \phi(t)^k \mathbb{P}(N=k) = g(\phi(t)). 
$$
Notice that the variance of $Y$ is related to its moments $\mathrm{Var}(Y) = m_2(Y) - m_1(Y)^2$, and that $m_r(Y) = (-i)^r \psi^{(r)}(0)$, so that $\mathrm{Var}(Y) = -\psi^{\prime\prime}(0)+\left( \psi^\prime(0)\right)^2$. Using $\psi = g \circ \phi$:
$$
  \psi^{\prime}(0)= g^\prime(1) \times \phi^\prime(0) = i \mathbb{E}(N) \mathbb{E}(X)
$$
and
$$
  \psi^{\prime\prime}(0)= g^{\prime\prime}(1) \phi^\prime(0)^2 + g^\prime(1) \phi^{\prime\prime}(0) = -\left( \mathbb{E}(X)^2 \cdot \mathbb{E}(N(N-1)) + \mathbb{E}(N) \cdot \mathbb{E}(X^2) \right)
$$
Combining these together will yield the result you seek to establish.
$$ \begin{eqnarray}
  \mathrm{Var}(Y) &=& \mathbb{E}(X)^2 \cdot \left( \mathbb{E}(N^2) -\mathbb{E}(N) \right) + \mathbb{E}(N) \cdot \left( \mathrm{Var}(X) + \mathbb{E}(X)^2 \right) - \mathbb{E}(N)^2 \cdot \mathbb{E}(X)^2 \\
  &=&  \mathbb{E}(X)^2 \cdot \left( \mathrm{Var}(N) + \mathbb{E}(N)^2 -\mathbb{E}(N) \right) + \mathbb{E}(N) \cdot \left( \mathrm{Var}(X) + \mathbb{E}(X)^2 \right) - \mathbb{E}(N)^2 \cdot \mathbb{E}(X)^2 \\
  &=& \mathbb{E}(X)^2 \cdot \mathrm{Var}(N) + \mathbb{E}(N) \cdot \mathrm{Var}(X)
 \end{eqnarray}
$$
Since I used $g^\prime(1) = \mathbb{E}(N)$ and $g^{\prime\prime}(1) = \mathbb{E}(N(N-1))$ I should note that they follow from the definition of the probability generating function 
$g(s) = \sum_{k=0}^\infty s^k \mathbb{P}(N=k)$.  Indeed $g^\prime(1) = \sum_{k=0}^\infty k \mathbb{P}(N=k) = \mathbb{E}(N)$, and $g^{\prime\prime}(1) = \sum_{k=0}^\infty k (k-1) \mathbb{P}(N=k) = \mathbb{E}(N (N-1))$.
