# How do you prove Cov $\left( \bar{X} , X_i - \bar{X} \right) = 0$?

How do you prove Cov $\left( \bar{X} , X_i - \bar{X} \right) = 0$ given $X_1 ,..., X_i$ are i.i.d. each with variance $\sigma^2$ and $\bar{X}$ is the sample mean?? In other words, how do you show that the sample mean and the differences of the observations from that mean are not linearly correlated???

Am I on the write track? I've written

$Var \left( \bar{X} + \left( X_i - \bar{X} \right) \right) = Var X_i = Var \bar{X} + Var \left( X_i - \bar{X} \right) + Cov \left( \bar{X} , X_i - \bar{X} \right)$ .

I know $Var X_i = \sigma^2 , Var \bar{X}= \frac{\sigma^2}{n}$, so all I need left to find the covariance is $Var \left( X_i - \bar{X} \right)$.

I don't know if this is right, but I wrote

$Var \left( X_i - \bar{X} \right) = E\left( \left( \left( X_i - \bar{X} \right) - \mu_{ X_i - \bar{X}} \right)^2 \right)=E \left( \left( X_i - \bar{X} \right)^2 \right)=s^2$, but I don't know if that's right or helpful at all.

Am I on the right track??

Hint: Use properties of the covariance function and independence of $X_1,...,X_n$.

Solution:

Note that $$Cov(\bar{X},X_i-\bar{X}) = Cov(\bar{X},X_i)-Cov(\bar{X},\bar{X}) = Cov(\bar{X},X_i)-Var(\bar{X})$$

You know that $Var(\bar{X})=\frac{\sigma^2}{n}$. So you need to find $Cov(\bar{X},X_i)$. But,

$$Cov(\bar{X},X_i)= Cov(\frac{1}{n}\sum_{j=1}^{n}X_j,X_i) = \frac{1}{n} \sum_{j=1}^{n} Cov(X_j,X_i)$$ Since $X_j$'s are independent it follows that $$Cov(X_j,X_i)=0 \text{ for all j } \neq i$$

So we conclude that $$Cov(\bar{X},X_i) = \frac{1}{n} Cov(X_i,X_i) = \frac{\sigma^2}{n}$$

It follows that $$Cov(\bar{X},X_i-\bar{X}) = 0$$

• Thank you so much I completely forgot about the property of Cov(aX+bY, cU +dV)!!!! You are absolutely a genius!!!
– ddd
Commented Sep 10, 2016 at 6:03
– Mdoc
Commented Sep 10, 2016 at 7:04

Note: This answer was originally posted in response to a duplicate Question, now closed. The first part is similar to the excellent answer of @Mdoc (+1). The second part discusses the importance of the result in statistical inference.

Residuals about a mean have $$0$$ covariance with the mean. Without loss of generality, find $$Cov(X_1-\bar X, \bar X):$$ Then $$Cov(X_1 - \bar X, \bar X) = Cov(X_1, \bar X) - Cov(\bar X,\bar X)\\ = Cov(X_1, \bar X) + Var(\bar X) = Cov(X_1,\bar X) -\sigma^2/n.$$

Now $$Cov(X_1,\bar X) = Cov\left(X_1, \frac 1n\sum_{i=1}^nX_i\right)\\ =Cov\left(X_1,\frac 1n X_1\right) + 0 = \frac 1n Cov(X_1,X_1)\\ = \frac 1n Var(X_1) = \sigma^2/n.$$

Thus, $$Cov(X_1,\bar X) = \sigma^2/n - \sigma^2/n = 0.$$

Relavance to statistical inference. This result is important in statistical inference. The residuals $$r_i = X_i - \bar X$$ of observations from their group means are widely used in ANOVA and regression.

Sample mean and variance independent for normal data. For normal data uncorrelated implies independent. Because $$\bar X$$ is independent of the $$r_i,$$ then it is independent of $$S.$$ So for normal data $$\bar X$$ and $$S_X^2$$ are stochastically independent. (They are not 'functionally' independent because $$\bar X$$ is used to find $$S_X^2.)$$ This is important for t statistics because Student's t distribution is defined in terms of a ratio with numerator and denominator independent.

Simulations illustrating lack of correlation. A brief simulation in R illustrates that means are not correlated with residuals from them. (The simulation uses 10 million normal samples of size $$n=10,$$ giving several decimal places of accuracy for the correlation.)

set.seed(2020)
M = 10^7; n = 10
X = rnorm(M*n, 100, 15)
DTA = matrix(X, nrow=M)
A = rowMeans(DTA)
X1 = DTA[,1]
cor(X1-A,A)
[1] -0.0004722208  # aprx 0


A similar simulation with exponential data also shows lack of correlation:

set.seed(2020)
M = 10^7; n = 10
Y = rexp(M*n)
DTA = matrix(Y, nrow=M)
A = rowMeans(DTA)
Y1 = DTA[,1]
cor(Y1-A,A)
[1] 4.620507e-08


However, scatterplots of residuals against means illustrates independence for the normal data, but a clear pattern of dependence for the exponential data. (We use reduced numbers of datasets for a manageable number of points in the scatterplots.)

m=30000
x1=X1[1:m]; a.x=A[1:m]; r.x=x1-a.x
y1=Y1[1:m]; a.y=A[1:m]; r.y=y1-a.y
par(mfrow=c(1,2))
plot(a.x,r.x, pch=".", main="Normal Data")
plot(a.y,r.y, pch=".", main="Exponential Data")
par(mfrow=c(1,1))