# Conditional Covariance of Functions of Random Variables

$\newcommand{\Cov}{\operatorname{Cov}} \newcommand{\E}{\mathbb{E}}$

I realize that $\Cov(X,Y) = \E[(X-\mu_X)(Y-\mu_Y)] = \E[\Cov(X,Y|A)] + \Cov(\E[X|A], \E[Y|A])$. But I am not sure how this is applied to functions of random variables conditioned on another random variable belonging to the same probability space.

Let $g(X,Y) = X / (X+Y)$ and $h(X,Y,Z) = (X+Y) / (X+Y+Z)$ , where $X, Y, Z$ are independent Poisson random variables with rates $\lambda_X$, $\lambda_Y$, and $\lambda_Z$, respectively, and where $X + Y + Z > 0$ and $g(X,Y) = 0$ if $Z=N$.

Suppose we know $N = X + Y + Z$.

What is the conditional covariance of $g(X,Y)$ and $h(X,Y,Z)$ given $N$ and how does one go about finding it?

Possibly this helps (?): For a known $N$, $h(X,Y,Z|N) \sim Binom(N,p) / N$ , where $p$ is the probability of success. This can be shown by recognizing $X+Y$ to be a sum of independent Poisson random variables, and thus also a Poisson random variable with rate $\lambda_X + \lambda_Y$. Conditioned on $(X+Y) + Z = N$, $(X+Y)$ is binomial with $p = (\lambda_X+\lambda_Y)/(\lambda_X + \lambda_Y + \lambda_Z)$.

-
How do you interpret $g(0,0)$ and $h(0,0,0)$? Or are you further conditioning that $X,Y,Z>0$? – Sasha Jul 2 '12 at 20:47
@Sasha Yes, sorry. I am further conditioning that $X+Y > 0$ and $X+Y+Z>0$. I have updated that post to reflect this condition. – kgryte Jul 3 '12 at 0:34
@MichaelHardy How does knowing (X+Y) is binomial completely answer the question? I don't quite see the connection to how $g(X,Y)$ and $h(X,Y,Z)$ covary given $N$. Perhaps you might be able to shed some light? Thanks. – kgryte Jul 3 '12 at 1:45
@kgryte : Sorry---my comment was hasty. – Michael Hardy Jul 3 '12 at 4:00
@MichaelHardy No worries. Maybe I am missing something, but the problem solution does not seem straightforward. And I have found little online in the way of worked examples to guide the process. Is the way forward sheets of algebra, or should some sort of propagation of errors technique be used? – kgryte Jul 3 '12 at 4:26

Based on the HINT from @did, here is a solution for the covariance of $g(X,Y|N)$ and $h(X,Y,Z|N)$.

$\DeclareMathOperator \Cov {Cov}$ $\DeclareMathOperator \E {E}$ $\DeclareMathOperator \Var {Var}$ $$\Cov(g(X,Y), h(X,Y,Z)|N) = \E\biggl[ g(X,Y)h(X,Y,Z)\bigg|N\biggr] - \E\biggl[g(X,Y)\bigg|N\biggr]\ \E\biggl[h(X,Y,Z)\bigg|N\biggr]$$

First consider $\E[X|N]$ $\ldots$

$$\E[X|X+Y+Z] = \E[X|X+W] = \frac{\lambda_X}{\lambda_X + \lambda_W} (X+W) = \frac{\lambda_X}{\lambda_X + \lambda_Y +\lambda_Z} (X+Y+Z) = \frac{\lambda_X}{\lambda_X + \lambda_Y + \lambda_Z} N$$

where we substituted having recognized that the sum of two independent Poisson random variables $Y+Z$ is also a Poisson random variable, say, $W$, whose rate is the sum $\lambda_Y + \lambda_Z$.

Now, consider the joint expectation $$\E\biggl[g(X,Y)h(X,Y,Z)\bigg|N\biggr] = \E\biggl[\frac{X}{X+Y+Z} \mathbf{1}_{X+Y+Z \neq 0} \bigg| X+Y+Z\biggr] \$$

We can use the property that $\E[aX] = a\E[X]\$, and hence,

$$\E\biggl[g(X,Y)h(X,Y,Z)\bigg|N\biggr] = \frac{\lambda_X}{\lambda_X+\lambda_Y+\lambda_Z} \mathbf{1}_{X+Y+Z \neq 0}$$

Now, consider $\E[X+Y|N]$ $\ldots$

$$E[X+Y|X+Y+Z] = \E[W|W+Z] = \frac{\lambda_W}{\lambda_W + \lambda_Z} (W+Z) = \frac{\lambda_X+\lambda_Y}{\lambda_X + \lambda_Y + \lambda_Z} N$$

where we made use of another $W$ substitution. Similar to above,

$$\E \biggl[h(X,Y,Z) \bigg| N \biggr] = \E\biggl[\frac{X+Y}{X+Y+Z} \mathbf{1}_{X+Y+Z \neq 0} \bigg| X+Y+Z \biggr] = \frac{\lambda_X+\lambda_Y}{\lambda_X+\lambda_Y+\lambda_Z} \mathbf{1}_{X+Y+Z \neq 0}$$

Now, one more term remains $\E\biggl[g(X,Y)\bigg| N\biggr]$. We can use the $Law\ of\ Iterated\ Expectations$

$$\E[X|N] = \E\biggl[\E[X|M]\bigg|N\biggr]$$

where the value of $N$ is determined by $M$. In our case, $N$ is determined by $Z$ and $X+Y$, both existing on the same probability space. So

$$\E\biggl[g(X,Y) \bigg| N \biggr] = \E \biggl[ \E [g(X,Y)|Z, X+Y)] \bigg| N\biggr]$$

But $g(X,Y)$ is independent of $Z$. In which case, focusing on the inner expectation,

$$\E \biggl[g(X,Y)\bigg|Z, X+Y)\biggr] = \E \biggl[g(X,Y)\bigg| X+Y) \biggr] = \E \biggl[ \frac{X}{X+Y} \mathbf{1}_{X+Y \neq 0} \bigg| X+Y \biggr]$$

where we have already seen this above.

$$\E \biggl[ \frac{X}{X+Y} \mathbf{1}_{X+Y \neq 0} \bigg| X+Y \biggr] = \frac{\lambda_X}{\lambda_X + \lambda_Y} \mathbf{1}_{X+Y \neq 0}$$

If we insert this result into the outer expectation, we have

$$\E \biggl[ g(X,Y) \bigg| N \biggr] = \E \biggl[ \frac{\lambda_X}{\lambda_X + \lambda_Y} \mathbf{1}_{X+Y \neq 0} \bigg| N \biggr] = \frac{\lambda_X}{\lambda_X + \lambda_Y} \mathbf{1}_{X+Y \neq 0}$$

where we used the property that the expectation of a constant is equal to the constant ($\E[b] = b$).

Putting the three terms together, we arrive at our solution for the covariance:

$$\Cov(g(X,Y), h(X,Y,Z) | N) = \frac{\lambda_X}{\lambda_X + \lambda_Y + \lambda_Z} - \biggl(\frac{\lambda_X}{\lambda_X + \lambda_Y} \cdot \frac{\lambda_X + \lambda_Y}{\lambda_X + \lambda_Y + \lambda_Z} \biggr) = 0$$

So while the functions $g(X,Y)$ and $h(X,Y,Z)$ are clearly dependent, they are not expected to co-vary.

-
Regarding the last term, the tower property shows that $E(g(X,Y)\mid N)=E(E(g(X,Y)\mid Z,X+Y)\mid N)$ and the independence of $Z$ and $(X+Y,g(X,Y))$ shows that $E(g(X,Y)\mid Z,X+Y)=E(g(X,Y)\mid X+Y)$. Hence... – Did Jul 3 '12 at 7:36
@did Thanks for the help. Any intuition as to why I should expect $g(\cdot)$ and $h(\cdot)$ to $\mathbf{not}$ co-vary? – kgryte Jul 3 '12 at 11:21
Yes: fix $N\geqslant1$, consider $N$ black balls, paint each black ball in red with probability $(\lambda_X+\lambda_Y)/\lambda$ with $\lambda=\lambda_X+\lambda_Y+\lambda_Z$, and re-paint each red ball in blue with probability $\lambda_X/(\lambda_X+\lambda_Y)$. Then the mean proportion of blue balls amongst the red balls is independent of the overall proportion of red balls. Now do this for $N$ random with distribution Poisson $\lambda$. The numbers of blue, red, and black balls are independent and Poisson with parameters $\lambda_X$, $\lambda_Y$ and $\lambda_Z$. – Did Jul 3 '12 at 16:26
@did Awesome! Thanks! – kgryte Jul 4 '12 at 2:12
@did How might we expect the covariance calculation to change in the case in which $X$ and $Z$ are dependent?And what might explain the situation in which $g(\cdot)$ and $h(\cdot)$ are observed to co-vary, i.e., what assumptions must be broken for the covariance to not equal 0? – kgryte Jul 4 '12 at 2:28

Hint: Independent Poisson random variables $U$ and $V$ with respective rates $u$ and $v$ are such that $\mathrm E(U\mid U+V)=\frac{u}{u+v}(U+V)$ hence $\mathrm E\left(\frac{U}{U+V}\,\mathbf 1_{U+V\ne0}\big\vert U+V\right)=\frac{u}{u+v}\,\mathbf 1_{U+V\ne0}$.

-
I counted three applications of the hint above to reach a solution. If you do not succeed in writing a proof, I might add some details. (Unrelated: you should really modify (or omit) the ${N\choose p}/N$ formula in the last paragraph of your post.) – Did Jul 3 '12 at 5:13
Thanks for your response. Why should the formula be modified? Is this not effectively the same formula you provided in your response for $\mathrm{E}(\frac{U}{U+V} | U+V)$ ? – kgryte Jul 3 '12 at 5:37
Because the LHS of the $\sim$ relation you wrote is a random variable and the RHS is either a number or a random variable while what you mean (and what the RHS should be) is a distribution. – Did Jul 3 '12 at 6:03
Right. I originally had $h(X,Y,Z|N) \sim Binom(N,p)/N$, but this was edited by someone else to its present form. – kgryte Jul 3 '12 at 6:30
I provided a partial solution for the covariance, but I am not sure about the '3rd' application of the hint. Any further advice for $\mathrm{E}[g(X,Y)|N]\$? Thanks again. – kgryte Jul 3 '12 at 7:09