# Covariance and Law of Large numbers

Say I am taking the average value of the product of two dependent random variables $$X$$ and $$Y$$ sampled an infinite amont of times. That is I am computing $$\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}X_{i}}{n} \right]$$. Is this the same as computing $$\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}}{n} \right] E \left[ \sum_{i=0}^{n} \frac{X_{i}}{n} \right] = E[X] E[Y]$$? Assuming $$X$$ and $$Y$$ have finite variance.

I know this would not be true if $$n$$ was small but does law of large numbers make the covariance $$0$$ in the same way it makes variance $$0$$?

• Are you sure? Isn't covariance a function of variance and variance goes to zero as n goes to infinity? – TPaul Dec 10 '18 at 22:06

From basic properties of expectation, $$E \sum_{i=0}^n \frac{Y_i X_i}{n} = \frac{1}{n} \sum_{i=0}^n E[X_i Y_i] = E[XY]$$ for every $$n$$. No limits, no law of large numbers.

If you also know $$X_i$$ and $$Y_i$$ are uncorrelated (e.g., if they are independent), then $$E[X_i Y_i] = E[X_i] E[Y_i]$$ and $$E[XY] = E[X] E[Y]$$.

Under the conditions of the law of large numbers (applied to $$XY$$), we have $$\sum_{i=0}^n \frac{Y_i X_i}{n} \to E[XY]$$ almost surely.

Again, if $$X$$ and $$Y$$ are uncorrelated, then $$E[XY] = E[X] E[Y]$$.

• My question is specifically about correlated values. The difference between $E[XY]$ and $E[X]E[Y]$ is a function of covariance. What does the law of large numbers say about covariance as n grows? Does it go to zero? How can i calculate it? – TPaul Dec 10 '18 at 22:11
• @TPaul If $X$ and $Y$ are zero mean, then the $\sum_{i=0}^n X_i Y_i/n$ tends to the covariance $\text{Cov}(X,Y) = E[XY]$ almost surely, as my answer implies. – angryavian Dec 10 '18 at 22:24
• Im sorry which part of your answer implies this? Also How does this answer the question i posed about covariance as n grows? – TPaul Dec 10 '18 at 22:45
I know this would not be true if n was small but does law of large numbers make the covariance 0 in the same way it makes variance 0?


Let me clarify this. LLN doesn't make the variance of the original random variables 0. It makes the variance of a random variable defined as sample average go to 0.

So $$X_n$$ would still have the same variance as $$X_1$$ as n goes to infinity. However if we define, $$Z = \left[ \sum_{i=0}^{n} \frac{X_{i}}{n} \right]$$, $$E[Z] = E[X_i]$$ and $$Var(Z) = \frac {Var(X)} {n}$$. As you see, when n goes to infinity, $$Var(Z)$$ goes to 0, but Var(X) doesn't change.

Coming back to the original question, $$\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}X_{i}}{n} \right] = E[XY]$$

$$E[XY] = E[X]E[Y] + cov(X,Y)$$

Since the $$cov(XY)$$ isn't going to change no matter the number of n, you cannot calculate the limit the way you proposed in question.

• I see. Thank you I think this is where i was stuck. – TPaul Dec 10 '18 at 23:02
• Does this imply $\lim_{n \rightarrow \infty} E[ \sum_{i=0}^{n} \frac{X_{i}}{n} \sum_{i=0}^{n} \frac{Y_{i}}{n}] = E[ \sum_{i=0}^{n} \frac{X_{i}}{n}] E[ \sum_{i=0}^{n} \frac{Y_{i}}{n}]$ ? – TPaul Dec 10 '18 at 23:05