# Prove $\bar{X}$ and $X_i-\bar{X}$ are independent if $X_i's$ are independently normally distributed

The statement comes from the book statistics and data analysis page 179, and I just wonder how to prove that $$\bar{X}$$ and $$X_i-\bar{X}$$ are independent if $$\require{enclose}\enclose{horizontalstrike}{Cov(\bar{X},X_i-\bar{X})=0}$$ and $$X_i's$$ are independently normally distributed.

• Is this what you're looking for?This may require $X_i$ to be i.i.d however. Commented Aug 31, 2022 at 1:24
• Thanks, but I'm trying to prove rigorously that they are independent given zero covariance. And yes I left out the details that $X_i$ is i.i.d normally distributed Commented Aug 31, 2022 at 1:47

(Note: In this answer I'll only consider standard normal $$X_i$$s, since it's no harder to do when they have other means and variances)

One way of doing this is with moment generating functions. For a bivariate random variable $$Z = (Z_1, Z_2)$$, $$Z_1$$ and $$Z_2$$ are independent if and only if the moment generating function $$M_Z(\mathbf{s}) = \mathbf{E} e^{\mathbf{s}^T Z}$$ factors into a product of a functions of $$s_1$$ and $$s_2$$ only, where $$\mathbf{s} = (s_1, s_2)$$.

Now, the thing to notice is that $$(\bar{X}, X_i - \bar{X})$$ is a linear transformation of the whole sample $$X = (X_1, \dotsc, X_n)$$. Namely, $$\begin{pmatrix}\bar{X} \\ X_i - \bar{X}\end{pmatrix} = \begin{pmatrix}\mathbf{1}^T/n \\ e_i^T - \mathbf{1}^T/n\end{pmatrix} X := H X$$ where $$\mathbf{1}$$ is a vector of all $$1$$s and $$e_i$$ is the vector of all $$0$$s with a $$1$$ in the $$i$$th entry.

We can use this to write down the MGF: \begin{align*} \mathbf{E} \exp\biggl\{\mathbf{s}^T \begin{pmatrix}\bar{X} \\ X_i - \bar{X}\end{pmatrix}\biggr\} = \mathbf{E} \exp\{\mathbf{s}^T H X\} = \mathbf{E} \exp\{(H^T\mathbf{s})^T X\} \end{align*}

Then we can use the known moment generating function of a vector of iid normals ($$\mathbf{E} e^{\mathbf{t}^T X} = \exp\{\frac{1}{2}\mathbf{t}^T \mathbf{t}\}$$) to conclude that $$\mathbf{E} \exp\{(H^T\mathbf{s})^T X\} = \exp\{\frac{1}{2}(H^T\mathbf{s})^T(H^T\mathbf{s})\} = \exp\{\frac{1}{2}\mathbf{s}^T HH^T \mathbf{s}\}.$$

Now, we can multiply \begin{align*} HH^T &= \begin{pmatrix}\mathbf{1}^T/n \\ e_i^T - \mathbf{1}^T/n\end{pmatrix} \begin{pmatrix}\mathbf{1}/n , & e_i - \mathbf{1}/n\end{pmatrix} \\ &= \begin{pmatrix} \mathbf{1}^T/n \mathbf{1}/n & \mathbf{1}^T/n (e_i - \mathbf{1}/n) \\ (e_i - \mathbf{1}/n)^T \mathbf{1}/n & (e_i - \mathbf{1}/n)^T(e_i - \mathbf{1}/n) \end{pmatrix} \\ &= \begin{pmatrix} 1/n & 0 \\ 0 & 1 - 1/n \end{pmatrix}, \end{align*} so that the MGF becomes $$\exp\biggl\{\frac{1}{2}\mathbf{s}^T HH^T \mathbf{s}\biggr\} = \exp\biggl\{\frac{1}{2}\biggl(\frac{1}{n}s_1^2 + \frac{n-1}{n}s_2^2\biggr)\biggr\} = e^{\frac{1}{2n}s_1^2} e^{\frac{n-1}{2n} s_2^2}.$$

Since this factors into terms containing only $$s_1$$ and $$s_2$$ respectively, we conlcude that $$\bar{X}$$ and $$X_i - \bar{X}$$ are independent (and also that they have $$N(0, 1/n)$$ and $$N(0, 1 - 1/n)$$ distributions respectively).

• This is really a Perfect answer! Thanks very much! Commented Aug 31, 2022 at 20:50