0
$\begingroup$

Say I am taking the average value of the product of two dependent random variables $X$ and $Y$ sampled an infinite amont of times. That is I am computing $\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}X_{i}}{n} \right]$. Is this the same as computing $\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}}{n} \right] E \left[ \sum_{i=0}^{n} \frac{X_{i}}{n} \right] = E[X] E[Y]$? Assuming $X$ and $Y$ have finite variance.

I know this would not be true if $n$ was small but does law of large numbers make the covariance $0$ in the same way it makes variance $0$?

$\endgroup$
  • $\begingroup$ Are you sure? Isn't covariance a function of variance and variance goes to zero as n goes to infinity? $\endgroup$ – TPaul Dec 10 '18 at 22:06
2
$\begingroup$

From basic properties of expectation, $$E \sum_{i=0}^n \frac{Y_i X_i}{n} = \frac{1}{n} \sum_{i=0}^n E[X_i Y_i] = E[XY]$$ for every $n$. No limits, no law of large numbers.

If you also know $X_i$ and $Y_i$ are uncorrelated (e.g., if they are independent), then $E[X_i Y_i] = E[X_i] E[Y_i]$ and $E[XY] = E[X] E[Y]$.


Under the conditions of the law of large numbers (applied to $XY$), we have $$\sum_{i=0}^n \frac{Y_i X_i}{n} \to E[XY]$$ almost surely.

Again, if $X$ and $Y$ are uncorrelated, then $E[XY] = E[X] E[Y]$.

$\endgroup$
  • $\begingroup$ My question is specifically about correlated values. The difference between $E[XY]$ and $E[X]E[Y]$ is a function of covariance. What does the law of large numbers say about covariance as n grows? Does it go to zero? How can i calculate it? $\endgroup$ – TPaul Dec 10 '18 at 22:11
  • $\begingroup$ @TPaul If $X$ and $Y$ are zero mean, then the $\sum_{i=0}^n X_i Y_i/n$ tends to the covariance $\text{Cov}(X,Y) = E[XY]$ almost surely, as my answer implies. $\endgroup$ – angryavian Dec 10 '18 at 22:24
  • $\begingroup$ Im sorry which part of your answer implies this? Also How does this answer the question i posed about covariance as n grows? $\endgroup$ – TPaul Dec 10 '18 at 22:45
1
$\begingroup$
I know this would not be true if n was small but does law of large numbers make the covariance 0 in the same way it makes variance 0?

Let me clarify this. LLN doesn't make the variance of the original random variables 0. It makes the variance of a random variable defined as sample average go to 0.

So $X_n$ would still have the same variance as $X_1$ as n goes to infinity. However if we define, $Z = \left[ \sum_{i=0}^{n} \frac{X_{i}}{n} \right]$, $E[Z] = E[X_i]$ and $Var(Z) = \frac {Var(X)} {n}$. As you see, when n goes to infinity, $Var(Z)$ goes to 0, but Var(X) doesn't change.

Coming back to the original question, $$\lim_{n \rightarrow \infty} E \left[ \sum_{i=0}^{n} \frac{Y_{i}X_{i}}{n} \right] = E[XY]$$

$$E[XY] = E[X]E[Y] + cov(X,Y)$$

Since the $cov(XY)$ isn't going to change no matter the number of n, you cannot calculate the limit the way you proposed in question.

$\endgroup$
  • $\begingroup$ I see. Thank you I think this is where i was stuck. $\endgroup$ – TPaul Dec 10 '18 at 23:02
  • $\begingroup$ Does this imply $\lim_{n \rightarrow \infty} E[ \sum_{i=0}^{n} \frac{X_{i}}{n} \sum_{i=0}^{n} \frac{Y_{i}}{n}] = E[ \sum_{i=0}^{n} \frac{X_{i}}{n}] E[ \sum_{i=0}^{n} \frac{Y_{i}}{n}]$ ? $\endgroup$ – TPaul Dec 10 '18 at 23:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.