1
$\begingroup$

Suppose we have identically distributed random variables with pairwise correlation $\rho$ and variance $\sigma^2$ (assume mean is zero). Then, defining $ \bar{X} = \frac{1}{n} \sum_{i=1}^n X_i, $ we have: \begin{equation} var(\bar{X}) = \mathbb{E} [\bar{X}^2] - \mathbb{E}[\bar{X}]^2 \end{equation}

\begin{align} \mathbb{E} [\bar{X}^2] &= \frac{1}{n^2} \mathbb{E} [\sum_i X_i^2 + \sum_i \sum_{j \neq i} X_i X_i] \\ &= \frac{1}{n^2} \{ \sum_i \mathbb{E} X_i^2 + \sum_i \sum_{j \neq i} \mathbb{E}[X_i X_j] \} \\ &= \frac{1}{n^2} \{ \sum_i \sigma^2 + \sum_i \sum_{j \neq i} \rho \sigma^2 \} \\ &= \frac{1}{n^2} \{ n \sigma^2 + n (n-1) \rho \sigma^2 \} \\ &= \frac{\sigma^2}{n} + \frac{n-1}{n} \rho \sigma^2 \end{align} and as the mean is zero, $\mathbb{E} [\bar{X}]^2 = 0$.

So, \begin{equation} var(\bar{X}) = \frac{\sigma^2}{n} + \frac{n-1}{n} \rho \sigma^2 \end{equation}

Now, suppose we have a fixed $n$. Then, if we generate identically distributed random variables with pairwise correlation $\rho = \frac{-1}{n-1}$, then \begin{equation} var(\bar{X}) = \frac{\sigma^2}{n} + \frac{n-1}{n} \sigma^2 (\frac{-1}{n-1}) = 0 \end{equation}

This seems super weird. I must be going wrong somewhere, but I don't know where! Would you please be able to have a look?

Thanks in advance!

$\endgroup$
0

1 Answer 1

1
$\begingroup$

There is nothing wrong with your work. It is just that you have an extreme (and so slightly weird) case. Note that the pairwise correlations cannot be less than $-\frac1{n-1}$ if you want the matrix to be positive semidefinite and even then there will be restrictions on the distribution of the $X_i$ to achieve this. The extreme effect is to make $\bar X=\mu$ $(=0$ here$)$ almost surely.

For example when $n=2$ this gives $\rho=-1$ and since $X_1$ and $X_2$ have identical distributions, you get $X_2=-X_1$ with probability $1$ and so $\bar X=0$, a constant with zero variance. For this to even be possible, you need the distribution of $X_1$ and $X_2$ to be symmetric about $0$.

Much the same thing happens with larger $n$. Here is a simulation using R of ten observations from a multivariate normal distribution and taking the correlation matrix as the covariance matrix when $n=6$:

library(mvtnorm)
set.seed(2022)
n <- 6
covmatrix <- matrix(rep(-1/(n-1), n^2), ncol=n) + diag(n)*n/(n-1) 
covmatrix
#      [,1] [,2] [,3] [,4] [,5] [,6]
# [1,]  1.0 -0.2 -0.2 -0.2 -0.2 -0.2
# [2,] -0.2  1.0 -0.2 -0.2 -0.2 -0.2
# [3,] -0.2 -0.2  1.0 -0.2 -0.2 -0.2
# [4,] -0.2 -0.2 -0.2  1.0 -0.2 -0.2
# [5,] -0.2 -0.2 -0.2 -0.2  1.0 -0.2
# [6,] -0.2 -0.2 -0.2 -0.2 -0.2  1.0
sims <- rmvnorm(10, mean=rep(0,n), sigma=covmatrix)
sims
#             [,1]        [,2]        [,3]        [,4]       [,5]        [,6]
# [1,]  2.05353690 -0.21785513  0.08433481 -0.51489124  0.7048736 -2.10999909
# [2,] -1.34855528  0.11628539  0.63282192  0.07644164  0.9140224 -0.39101608
# [3,] -0.59594809  0.58136474  0.42176678  0.39159438 -0.2369455 -0.56183242
# [4,]  0.21283317  0.03699741 -0.50479400 -0.48377217  0.3156341  0.42310169
# [5,] -0.20142159 -0.76144379  0.89221894  0.53952061 -0.3872322 -0.08164199
# [6,]  0.23320763  1.01507018 -1.60352276 -0.13737918 -0.7910043  1.28362840
# [7,]  0.86343020 -0.16358892 -0.14731998 -0.50410625 -0.6139985  0.56558351
# [8,] -0.64649148 -0.98177522  0.65169196 -0.28309427  0.9583281  0.30134078
# [9,] -2.00950883 -0.17680495  1.27467798  0.51076605 -0.3755420  0.77641190
#[10,] -0.04605967  0.31143296 -0.35956513  0.90532379 -0.8817965  0.07066465
Xbar <- rowMeans(sims)
Xbar
# [1] -3.246942e-08  5.724417e-09 -1.458760e-08  2.749809e-08 -5.479594e-09
# [6] -3.169053e-09  9.711573e-09 -1.781134e-08  1.764540e-08  1.170931e-08
var(Xbar)
# [1] 3.284356e-16

so neither the simulated $\bar X$s nor their variance are precisely $0$ but close enough to make the point.

$\endgroup$
2
  • $\begingroup$ Wow, I find this very interesting. Thank you very much for the detailed explanation. Would you happen to know whether this property has a name that I could look for in literature? $\endgroup$ Commented Feb 16, 2022 at 1:02
  • 1
    $\begingroup$ I do not know about a name. In effect the correlation matrix is positive semidefinite but not positive definite so is a singular matrix and cannot be inverted, which would undermine the density of such a distribution, and indeed there is no density as the support is not the whole space: for $n=3$ all the points lie on the plane $X_1+X_2+X_3=0$. Perhaps it might be called an extreme correlation matrix. $\endgroup$
    – Henry
    Commented Feb 16, 2022 at 1:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .