# Variance of sample mean of correlated random variables is zero?

Suppose we have identically distributed random variables with pairwise correlation $$\rho$$ and variance $$\sigma^2$$ (assume mean is zero). Then, defining $$\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i,$$ we have: $$$$var(\bar{X}) = \mathbb{E} [\bar{X}^2] - \mathbb{E}[\bar{X}]^2$$$$

\begin{align} \mathbb{E} [\bar{X}^2] &= \frac{1}{n^2} \mathbb{E} [\sum_i X_i^2 + \sum_i \sum_{j \neq i} X_i X_i] \\ &= \frac{1}{n^2} \{ \sum_i \mathbb{E} X_i^2 + \sum_i \sum_{j \neq i} \mathbb{E}[X_i X_j] \} \\ &= \frac{1}{n^2} \{ \sum_i \sigma^2 + \sum_i \sum_{j \neq i} \rho \sigma^2 \} \\ &= \frac{1}{n^2} \{ n \sigma^2 + n (n-1) \rho \sigma^2 \} \\ &= \frac{\sigma^2}{n} + \frac{n-1}{n} \rho \sigma^2 \end{align} and as the mean is zero, $$\mathbb{E} [\bar{X}]^2 = 0$$.

So, $$$$var(\bar{X}) = \frac{\sigma^2}{n} + \frac{n-1}{n} \rho \sigma^2$$$$

Now, suppose we have a fixed $$n$$. Then, if we generate identically distributed random variables with pairwise correlation $$\rho = \frac{-1}{n-1}$$, then $$$$var(\bar{X}) = \frac{\sigma^2}{n} + \frac{n-1}{n} \sigma^2 (\frac{-1}{n-1}) = 0$$$$

This seems super weird. I must be going wrong somewhere, but I don't know where! Would you please be able to have a look?

There is nothing wrong with your work. It is just that you have an extreme (and so slightly weird) case. Note that the pairwise correlations cannot be less than $$-\frac1{n-1}$$ if you want the matrix to be positive semidefinite and even then there will be restrictions on the distribution of the $$X_i$$ to achieve this. The extreme effect is to make $$\bar X=\mu$$ $$(=0$$ here$$)$$ almost surely.

For example when $$n=2$$ this gives $$\rho=-1$$ and since $$X_1$$ and $$X_2$$ have identical distributions, you get $$X_2=-X_1$$ with probability $$1$$ and so $$\bar X=0$$, a constant with zero variance. For this to even be possible, you need the distribution of $$X_1$$ and $$X_2$$ to be symmetric about $$0$$.

Much the same thing happens with larger $$n$$. Here is a simulation using R of ten observations from a multivariate normal distribution and taking the correlation matrix as the covariance matrix when $$n=6$$:

library(mvtnorm)
set.seed(2022)
n <- 6
covmatrix <- matrix(rep(-1/(n-1), n^2), ncol=n) + diag(n)*n/(n-1)
covmatrix
#      [,1] [,2] [,3] [,4] [,5] [,6]
# [1,]  1.0 -0.2 -0.2 -0.2 -0.2 -0.2
# [2,] -0.2  1.0 -0.2 -0.2 -0.2 -0.2
# [3,] -0.2 -0.2  1.0 -0.2 -0.2 -0.2
# [4,] -0.2 -0.2 -0.2  1.0 -0.2 -0.2
# [5,] -0.2 -0.2 -0.2 -0.2  1.0 -0.2
# [6,] -0.2 -0.2 -0.2 -0.2 -0.2  1.0
sims <- rmvnorm(10, mean=rep(0,n), sigma=covmatrix)
sims
#             [,1]        [,2]        [,3]        [,4]       [,5]        [,6]
# [1,]  2.05353690 -0.21785513  0.08433481 -0.51489124  0.7048736 -2.10999909
# [2,] -1.34855528  0.11628539  0.63282192  0.07644164  0.9140224 -0.39101608
# [3,] -0.59594809  0.58136474  0.42176678  0.39159438 -0.2369455 -0.56183242
# [4,]  0.21283317  0.03699741 -0.50479400 -0.48377217  0.3156341  0.42310169
# [5,] -0.20142159 -0.76144379  0.89221894  0.53952061 -0.3872322 -0.08164199
# [6,]  0.23320763  1.01507018 -1.60352276 -0.13737918 -0.7910043  1.28362840
# [7,]  0.86343020 -0.16358892 -0.14731998 -0.50410625 -0.6139985  0.56558351
# [8,] -0.64649148 -0.98177522  0.65169196 -0.28309427  0.9583281  0.30134078
# [9,] -2.00950883 -0.17680495  1.27467798  0.51076605 -0.3755420  0.77641190
#[10,] -0.04605967  0.31143296 -0.35956513  0.90532379 -0.8817965  0.07066465
Xbar <- rowMeans(sims)
Xbar
# [1] -3.246942e-08  5.724417e-09 -1.458760e-08  2.749809e-08 -5.479594e-09
# [6] -3.169053e-09  9.711573e-09 -1.781134e-08  1.764540e-08  1.170931e-08
var(Xbar)
# [1] 3.284356e-16


so neither the simulated $$\bar X$$s nor their variance are precisely $$0$$ but close enough to make the point.

• Wow, I find this very interesting. Thank you very much for the detailed explanation. Would you happen to know whether this property has a name that I could look for in literature? Commented Feb 16, 2022 at 1:02
• I do not know about a name. In effect the correlation matrix is positive semidefinite but not positive definite so is a singular matrix and cannot be inverted, which would undermine the density of such a distribution, and indeed there is no density as the support is not the whole space: for $n=3$ all the points lie on the plane $X_1+X_2+X_3=0$. Perhaps it might be called an extreme correlation matrix. Commented Feb 16, 2022 at 1:17