Given identically distributed random variables $X_1,...,X_n$, where $ \mathbb E X_i = 0$, $\mathbb E [X_i^2] = 1$, and $\mathbb E [X_i X_j] = c<1$, define $S_n = X_1 + ... + X_n$. Will

$$\frac{S_n}{n} \rightarrow 0$$ in probability as $n\rightarrow \infty$?

The closest question I could find is this one, where an additional constraint is placed on the covariances, such that Chebyshev inequality can be used to bound $\left|\frac{S_n}{n} \right|$ by $\text{Var}\left(\frac{S_n}{n}\right)$. However, in my case $\text{Var}\left(\frac{S_n}{n}\right)$ is constant, so this approach will not work.

Indeed, when I simulate using MATLAB on an example where I generate Gaussian random variables which all have covariance $c=0.1$, I do not see it approaching zero.

NN =20000;
C = ones(NN,NN)*0.1 + 0.9*diag(ones(NN,1));
rndnums = mvnrnd(zeros(NN,1), C, 1);
NNs = 1:20:NN
means = -99*ones(1,length(NNs));
for ti=1:length(NNs)
    means(ti) = mean(rndnums(1,1:NNs(ti)));
scatter(NNs ,means, '.'); xlabel('n'); ylabel('mean')

enter image description here

Intuitively, since all the variables are all "locked" to eachother in correlation, I can't imagine how their mean will "eventually" reach zero.

Is there a law of large numbers in this context? If so, what is the rate of convergence? Or if not, is there any way to tell what it will converge to?

  • 1
    $\begingroup$ How can you show that $c \leq \frac1{n-1}$? What is wrong, then, with the OP's example, where it looks like $c$ can be 0.1 regardless of how large $n$ is? $\endgroup$ Commented May 9, 2019 at 20:19
  • $\begingroup$ @kimchilover Not true. The $n \times n$ matrix with diagonal entries $1$ and other entries $c$ has eigenvalues $(n-1) c + 1$ and $1-c$ (with multiplicity $n-1$), and thus is positive definite if $-1/(n-1) < c < 1$. $\endgroup$ Commented May 9, 2019 at 20:32
  • $\begingroup$ @RobertIsrael, and others: I'm sorry; I mixed up signs. What I should have said was $1+(n-1)c\ge0$, that is, $c\ge -1/(n-1)$ (which is implicit in RobertIsrael's construction). $\endgroup$ Commented May 9, 2019 at 20:35
  • $\begingroup$ If $X_1, \ldots, X_n$ are jointly normal with means $0$ and covariance matrix having diagonal entries $1$ and all other entries $c$, you can define $X_{n+1}$ as a suitable linear combination of $X_1 + \ldots + X_n$ and an independent normal random variable so that $X_1, \ldots, X_{n+1}$ are iid and satisfy the condition. $\endgroup$ Commented May 9, 2019 at 20:37
  • $\begingroup$ Of course, in the jointly normal case $S_n/n$ is normal with mean $0$ and variance $c + (1-c)/n$, and so it doesn't go to $0$ in probability. I don't know if this is true in general. $\endgroup$ Commented May 9, 2019 at 20:43

1 Answer 1


Letting $A$ be the ($n\times n$) matrix with ones in all entries, and $I$ be the $n\times n$ identity matrix, the covariance matrix is $$ M = (1-c)I + cA = P D P^{-1} $$ where $D$ is diagonal and $P$ is orthogonal, and (as @RobertIsrael observed) we are free to order the eigenvalues such that the upper left element of $D$ is $1+(n-1)c$ and the remaining diagonal elements are $1-c$.

When we do this, we find that $(1,1,1,\ldots,1) P = (\sqrt{n},0,0,0...)$ which tells us that the linear combinations of the variates that do not constitute the eigenvector with eigenvalue $(1-c)$ do not contribute to the sum $S_n$.

So the sum $S_n$ is distributed as $\sqrt{n}$ times a variate with mean zero (of course) and variance $1+(n-1)c$. This need not be a Gaussian but if the tails are not thick (so that the law of large numbers applies) then for large $n$ the standard deviation of $S_n/n$ goes like $$ \frac{\sqrt{n}\sqrt{1+(n-1)c}}{n} \sim \sqrt{c} $$ So $S_n/n$ will not approach zero with probability one.

I think even if the individual variates have thick tails, that only makes matters worse for saying that $S_n/n$ will approach zero with probability one, so the above statement still holds.

  • $\begingroup$ Thanks for the answer. It is not immediately obvious to me why the eigenvalues of the covariance matrix are (1+(n-1)c, 1-c, 1-c, ...., 1-c). Actually, I also don't follow the reasoning about the eigenvector with eigenvalue (1-c) either, but I'm sure that's because I'm missing something in the first part. $\endgroup$ Commented May 12, 2019 at 1:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .