# Intuition wrong? Variance of two related sums of random variables.

I recently have been trying to analyze the variance of two related random variables $G$ and $F$, hoping to get a decrease of the variance when switching to the second ($F$). However, either my computations are wrong, or my intuition is -- it looks like the variance goes up. At this point, I am hoping for either finding a mistake (it goes down) or understanding why.

Let $(p_1,\dots,p_n)\in[0,1]^n$ be parameters such that $q_i=q\stackrel{\rm{}def}{=}\frac{1}{n}\sum_{i=1}^n p_i \in (0,1)$ for all $i\in[n]$. Let $X_1,\dots, X_n$ be independent Poisson random variables, where $X_i\sim\operatorname{Poisson}(p_i)$; and $Y_1,\dots,Y_n$ be independent random variables (also independent from the $X_i$'s) with $Y_i\sim\operatorname{Poisson}(q_i)$.

I also have a random variable $Y\sim\operatorname{Poisson}(q)$, independent of $(X_i)_{1\leq i \leq n}$. ($Y$ and all the $Y_i$'s are identically distributed).

Let $$G\stackrel{\rm{}def}{=} \sum_{i=1}^n \left( (X_i-Y_i)^2 - X_i - Y_i \right)$$ and $$F\stackrel{\rm{}def}{=} \sum_{i=1}^n \left( (X_i-Y)^2 - X_i - Y \right).$$

We have* $\mathbb{E} G = \mathbb{E} F = \sum_{i=1}^n (p_i-q_i)^2$. What I am interested in is the (way messier to compute for $F$, as the terms of the sums are no longer independent) variance of both $F$ and $G$; getting closed form expression, or at least upper bounding them. I have this for $G$, but "intuitively" feel that one should get $$\operatorname{Var} F \leq \operatorname{Var} G \tag{\dagger}$$ as instead of many $Y_i$ which induce fluctuations, there is only one random variable $Y$. Trying to compute it, using properties of the Poisson distribution and breaking the variance into $\mathbb{E}[ G^2] - (\mathbb{E} G)^2$ (and expanding the square of the sum in the expectation of the first term) is (a) horrendous; and (b) the result I got seems to contradict my "conjecture" $(\dagger)$.

Is there a simple way to prove/disprove what I thought, or (even better) to elegantly compute the variance to do so?

Best,

Clément.

*(unless I messed up with the statement of the problem)

• There might be a misprint in the definition of $q_i$ (the RHS does not depend on $i$). Or do you mean $q_i=\frac1n\sum\limits_{k=1}^nq_k$? – Did Jun 20 '14 at 19:24
• Sometimes using multiple independent and identically distributed samples will reduce variance. The most famous example is that $\frac{1}{n}\sum_{i=1}^nX_i$ has a smaller variance than $X_1$, even thought it has the same mean. – Michael Jun 20 '14 at 19:27
• You should be able to get closed form answers for both of these. We have $Var(F) = E[F^2] - E[F]^2$. You already know $E[F]$. – Michael Jun 20 '14 at 19:31
• @Michael: I know, I did it -- and was puzzled by the result. Hence the question, to understand it or find out if I had made a mistake. – Clement C. Jun 23 '14 at 21:57
• @Did: the quantity $q_i$ is a constant (but I do have a conflict of notation in my question with $i$ also being the summation -- dummy -- variable). $q_i=q$ is the value obtained by "spreading uniformly" the total $p([n])$ on the $n$ elements. – Clement C. Jun 23 '14 at 21:59

\begin{eqnarray*} G_i &=& (X_i-Y_i)^2 - (X_i+Y_i)\\ F_i &=& (X_i-Y)^2 - (X_i+Y) \end{eqnarray*} Then $G=\sum_{i=1}^n G_i$ and $F = \sum_{i=1}^nF_i$. The $\{G_1, \ldots, G_n\}$ variables are independent, while the $\{F_1, \ldots, F_n\}$ variables are not. However, for each $i \in \{1, \ldots, n\}$, $F_i$ and $G_i$ are identically distributed. So $Var(F_i)=Var(G_i)$ and:
$Var(G) = \sum_{i=1}^nVar(G_i)$
$Var(F) = \sum_{i=1}^nVar(G_i) + \sum_{i\neq j} E[(F_i-E[F_i])(F_j-E[F_j])]$
Thus, $Var(F)>Var(G)$ when the $F_i$ variables are postively correlated with each other, as they probably are in this case.
You can reduce variance if you find a way of making $F_i$ samples that are negatively correlated with each other.
• Related to Did's comment: I assume you mean that $q_i=q$ for all $i$, so $\{Y_1, \ldots, Y_n\}$ are i.i.d. random variables. My answer above assumes that. – Michael Jun 20 '14 at 21:00