Let X1, X2... be iid Bernoulli random variables with parameter 1/4, let Y1, Y2... be another sequence of iid Bernoulli random variables with parameter 3/4 and let N be a geometric random variable with parameter 1/2. Assume the Xi's, Yj's and N are all independent. Compute Covariance(Σ Xi,Σ Yi), i is from 1 to N.

Since X and Y are independent random variables, I concluded that Covariance of the two terms is 0. However, I strongly feel that the mixed distribution has a part to play here. Since both the distributions have n as a common parameter, can they still be considered independent?

  • $\begingroup$ To summarise, you are asking us to find the covariance of two independent random variables? $\endgroup$ – Angina Seng Jun 24 at 2:33
  • $\begingroup$ X and Y both have n as a common parameter, can they still be considered independent? @AnginaSeng. Okay, I think I should post the exact question here. Let me edit it $\endgroup$ – Nisha Jun 24 at 2:35
  • $\begingroup$ What are $X$ and $Y$? Sums of $X_i$ and $Y_i$? In any case sharing n is irrelevant, so co-variances are $0$. $\endgroup$ – herb steinberg Jun 24 at 3:02

This is one of those situations where it is clearer to write the limits of summation: let $$S = \sum_{i=1}^N X_i, \quad T = \sum_{i=1}^N Y_i.$$ Then they certainly have positive covariance because they both depend on $N$ in the same way. Had the upper limits been fixed, say at $5$ for $X_i$ and $11$ for $Y_i$, then the covariance is zero. But as the question is framed, knowledge of $S$, for instance, carries some information about $N$ when it is unknown, hence informs the value of $T$; e.g., observing $S = 5$ means $N \ge 5$.

To compute the covariance, it is useful to employ the law of total expectation. We first compute $$\operatorname{E}[S] = \operatorname{E}[\operatorname{E}[S \mid N]] = \operatorname{E}[N p_x] = \frac{p_x}{\theta},$$ where I have taken $p_x$ to be the Bernoulli parameter for $X_i$, and I am using the parametrization $$\Pr[N = n] = (1-\theta)^{n-1} \theta, \quad n \in \{1, 2, \ldots\},$$ so $\theta$ is the geometric distribution parameter. Similarly, $$\operatorname{E}[T] = \frac{p_y}{\theta}.$$ We now turn our attention to the computation of $$\operatorname{E}[ST] = \operatorname{E}[\operatorname{E}[ST \mid N]].$$ Note that since $N$ is given, the inner expectation is simply $$\operatorname{E}[ST \mid N] = \operatorname{E}[S \mid N]\operatorname{E}[T \mid N] = Np_x Np_y = N^2 p_x p_y.$$ Then the outer expectation with respect to $N$ is $$\operatorname{E}[ST] = p_x p_y \operatorname{E}[N^2] = \frac{2-\theta}{\theta^2} p_x p_y.$$ Consequently, $$\operatorname{Cov}[S,T] = \operatorname{E}[ST] - \operatorname{E}[S]\operatorname{E}[T] = \frac{2-\theta}{\theta^2} p_x p_y - \frac{p_x p_y}{\theta^2} = \frac{1-\theta}{\theta^2} p_x p_y.$$ It is important to understand here that this is the unconditional covariance of $S$ and $T$, rather than $$\operatorname{Cov}[S \mid N, T \mid N] = 0.$$

Had we used the alternative parametrization for $N$ $$\Pr[N = n] = (1-\theta)^n \theta, \quad n \in \{0, 1, 2, \ldots\},$$ then we would need to adjust accordingly to obtain $$\operatorname{Cov}[S,T] = \frac{2(1-\theta)^2}{\theta^2} p_x p_y.$$

Here is Mathematica code to simulate the covariance in the case $p_x = 1/4$, $p_y = 3/4$, $\theta = 1/2$ as stated in the question, with the number of simulations equal to $10^6$:

d = ParallelTable[{RandomVariate[BinomialDistribution[#, 1/4]], 
    RandomVariate[BinomialDistribution[#, 3/4]]} 
    &[RandomVariate[GeometricDistribution[1/2]] + 1], {10^6}];

Mean[Times @@ # & /@ d] - Times @@ (Mean /@ Transpose[d]) // N

This gave me a result equal to $0.375653$, which is close to the theoretical result $3/8$. The same can be done with the alternate parametrization by removing the +1. It just so happens that for the given choice of parameters, the covariance is again $3/8$, but when $\theta \ne 1/2$, the two covariances will not be the same.

| cite | improve this answer | |
  • $\begingroup$ Yes, thankyou for making the point clear as to why the covariance is not 0. $\endgroup$ – Nisha Jun 24 at 6:04
  • $\begingroup$ Could you also explain this step: Note that since N is given, the inner expectation is simply E[ST∣N]=E[S∣N]E[T∣N]=NpxNpy=N2pxpy. How do you conclude that the two random variables are independent? $\endgroup$ – Nisha Jun 24 at 6:04
  • $\begingroup$ @Nisha the unconditional random variables $S$ and $T$ are not independent as they both depend on $N$. But once $N$ is fixed, the conditional random variables $S \mid N$ and $T \mid N$ are independent because the $X_i$s and $Y_i$s are independent. The link between the outcomes of $S$ and $T$ is broken once you fix $N$ in advance. This is why $\operatorname{E}[ST \mid N] = \operatorname{E}[S \mid N]\operatorname{E}[T \mid N]$, but $\operatorname{E}[ST] \ne \operatorname{E}[S]\operatorname{E}[T]$. $\endgroup$ – heropup Jun 24 at 6:11
  • $\begingroup$ All points clarified. Thankyou! $\endgroup$ – Nisha Jun 24 at 6:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.