On Wikipedia Strong Law of Large Numbers is given as follows.

Let $\{X_n\}$ be a sequence of independent identically distributed $\mathbb{R}$-valued random variables with finite expectation $\mathrm{E}[X_k] = \mu, ~ \forall k \in \mathbb{N}$. Let $S_n$ denote the corresponding sequence of partial sums. Then as $n\rightarrow\infty$ we have $\frac{S_n}{n}\rightarrow \mu$ almost surely.

My question: Is it possible to weaken the condition of $\{X_n\}$ beeing independent identically distributed (i.i.d) random variables and instead merely consider pairwise uncorrelated identically distributed $\{X_n\}$?

In 1980 Etemadi proved that i.i.d random variables can be replaced by pairwise independent random variables in the aforecited SLLN. I know that pairwise independence concept is close to pairwise uncorrelatedness. On the other hand $\mathrm{Cov}(X_i,X_j)$ by definition requires random variables $X_i$ and $X_j$ to have finite variances. Does it mean that answer to my question is "NO" because $X_i$ and $X_j$ have only finite expectations?

  • $\begingroup$ By the way, does it mean that independence between random variables implies their uncorrelatedness only if they fave finite variances? (otherwise covariance will be undefined)? $\endgroup$ – Rodvi Nov 30 '17 at 22:52
  • 1
    $\begingroup$ I may have left out a condition: If the variables $\{X_i\}$ are lower bounded by some constant (as in always nonnegative) and are pairwise uncorrelated with finite variance, then the LLN holds. I'm not sure if the "lower bounded" restriction can be removed. (It can be removed if the variables are independent). $\endgroup$ – Michael Nov 30 '17 at 23:00
  • 1
    $\begingroup$ If $X$, $Y$ are independent and have finite means then $E[XY]=E[X]E[Y]$, and I would call that "uncorrelated" even if variances are infinite. I'm not sure why wikipedia requires finite variance. $\endgroup$ – Michael Nov 30 '17 at 23:18

There is likely a proof somewhere on this site but I could not find it. Here I give a quick proof of my comment (since I originally mis-stated the result by forgetting the "lower bounded" restriction):

Let $\{X_i\}_{i=1}^{\infty}$ be a sequence of random variables, not necessarily identically distributed and not necessarily independent, that satisfy:

i) $E[X_i]=m_i$, where $m_i \in \mathbb{R}$ for all $i\in\{1, 2, 3, ...\}$.

ii) There is a constant $\sigma^2_{bound}$ such that $Var(X_i) \leq \sigma^2_{bound}$ for all $i \in \{1, 2, 3, ...\}$.

iii) The variables are pairwise uncorrelated, so $E[(X_i-m_i)(X_j-m_j)]=0$ for all $i \neq j$.

iv) There is a value $b \in \mathbb{R}$ such that, with prob 1, $X_i-m_i\geq b$ for all $i \in \{1, 2, 3, ...\}$.

Define $L_n = \frac{1}{n}\sum_{i=1}^n (X_i-m_i)$. Then $L_n\rightarrow 0$ with prob 1.

Proof: Since the variables are pairwise uncorrelated with bounded variance, we easily find for all $n$:

$$ E[L_n^2] = \frac{1}{n^2}\sum_{i=1}^n \sigma_i^2 \leq \frac{\sigma_{bound}^2}{n} $$

Fix $\epsilon>0$. It follows that: $$ P[|L_n|>\epsilon] = P[L_n^2 \leq \epsilon^2] \leq \frac{E[L_n^2]}{\epsilon^2} \leq \frac{\sigma_{bound}^2}{n\epsilon^2} $$ Hence: $$ \sum_{n=1}^{\infty} P[|L_{n^2}|>\epsilon] \leq \sum_{n=1}^{\infty}\frac{\sigma_{bound}^2}{n^2\epsilon^2} < \infty $$ and so $L_{n^2}\rightarrow 0$ with probability 1 by the Borel-Cantelli Lemma. That is, the $L_n$ values converge over the sparse subsequence $n\in\{1, 4, 9, 16, ...\}$.

Since $L_n \geq b$ for all $n$ and $L_{n^2}\rightarrow 0$ with probability 1, it can be shown that $L_n\rightarrow 0$ with probability 1. $\Box$

The lower bounded condition is typically treated by writing $X_n = X_n^+ - X_n^-$ where $X_n^+$ and $X_n^-$ are nonnegative and defined $X_n^+=\max[X_n,0]$, $X_n^-=-\min[X_n,0]$. If $X_n$ and $X_i$ are independent, then $X_n^+$ and $X_i^+$ are also independent. So the lower bounded condition can be removed for the case when variables are independent. However, if $X_n$ and $X_i$ are uncorrelated, that does not mean $X_n^+$ and $X_i^+$ are uncorrelated. So it is not clear to me if the lower-bounded condition can be removed when "independence" is replaced by the weaker condition "pairwise uncorrelated."

| cite | improve this answer | |
  • 1
    $\begingroup$ Okay, the lower-bound condition can be removed by repeating the proof to show that $L_{\lceil n^{1.5}\rceil}\rightarrow 0$ with prob 1, observing that $P[|X_i-m|>k^{.8}]\leq \frac{\sigma^2_{bound}}{k^{1.6}}$, and using Borel-Cantelli to show that with prob 1, for sufficiently large $k$, all $|X_i-m|$ values for $i \in \{\lceil k^{1.5}\rceil, ...,\lceil (k+1)^{1.5}\rceil\}$ are less than or equal to $k^{.8}$. $\endgroup$ – Michael Dec 1 '17 at 1:08
  • $\begingroup$ Good job! How do you think, should we require finite variances in the abovementioned LLN if we replace a.s. convergence by convergence in probability (i.e. for WLLN case)? $\endgroup$ – Rodvi Dec 1 '17 at 10:59
  • 1
    $\begingroup$ If you only want convergence in probability then simplifying the above proof shows that it is enough for $\{X_i\}$ to have finite variances $\sigma_i^2$, $Cov(X_i,X_j)\leq 0$ for all $i \neq j$, and $\lim_{n\rightarrow\infty} \frac{1}{n^2}\sum_{i=1}^n \sigma_i^2 = 0$. A counter-example when variances increase too quickly is when $\{Y_i\}$ are iid uniform $[-1,1]$ and $X_i= iY_i$. So $E[X_i]=0$, $Cov(X_i,X_j)=0$ for $i\neq j$, but $\frac{1}{n}\sum_{i=1}^n X_i$ does not converge to 0 in probability. $\endgroup$ – Michael Dec 1 '17 at 19:05
  • 1
    $\begingroup$ If $\{X_i\}$ are identically distributed with finite mean and infinite variance, and they are pairwise uncorrelated, I cannot think of a counter-example, but I also do not know how to prove LLN holds. It certainly holds if "pairwise uncorrelated" is replaced by "pairwise independent" since that is the standard LLN statement, and we can do truncation tricks ("independence" is preserved after truncation, while "uncorrelated" is not necessarily preserved). $\endgroup$ – Michael Dec 1 '17 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.