# Prove $\operatorname{Cov}(\overline{X_n}, X_j - \overline{X_n}) = 0$ for independent normally distributed random variables

My homework states the following problem:

Let $X_1, \dots, X_n$ be independent $N(\mu, \sigma^2)$ distributed random variables, $\overline{X_n}$ be the sample mean and $S_n^2$ the empirical variance. Show that $\operatorname{Cov}(\overline{X_n}, X_j - \overline{X_n}) = 0$ and conclude that $\overline{X_n}$ and $S_n^2$ are independent.

My first approach:

\begin{align*} \operatorname{Cov}(\overline{X_n}, X_j - \overline{X_n}) &= \mathbb E(\overline{X_n} X_j - \overline{X_n}^2) - \mathbb E(\overline{X_n}) \mathbb E(X_j - \overline{X_n}) \\ &= \mathbb E(\overline{X_n} X_j) - \mathbb E(\overline{X_n}^2) - \mathbb E(\overline{X_n}) \mathbb E(X_j - \overline{X_n}) \\ \end{align*}

My second approach is

\begin{align*} \operatorname{Cov}(\overline{X_n}, X_j - \overline{X_n}) &= \mathbb E\left[(\overline{X_n} - \mathbb E(\overline{X_n})) (X_j - \overline{X_n} - \mathbb E(X_j - \overline{X_n}))\right] \end{align*}

I always end up with expressions involving the sample mean and the population mean. My problem is: I know about the Law of Large Numbers bridging the gap between samples and populations, but here no series is given, but finite $n$. Which theorem can help me to solve this problem?

Update: A solution to the second question is given at jekyll.math.byuh.edu

You have \begin{align}\text{Cov}(\bar X_n,X_j-\bar X_n)&=\text{Cov}(X_j,\bar X_n)-\text{Var}(\bar X_n)\\&=\text{Cov}\left(X_j,\frac{1}{n}\sum_{i=1}^nX_i\right)-\text{Var}(\bar X_n)\\&=\frac{1}{n}\sum_{i=1}^n\text{Cov}(X_i,X_j)-\text{Var}(\bar X_n)\\&=\frac{1}{n}\left(\text{Var}(X_i)+\sum_{i\ne j}\text{Cov}(X_i,X_j)\right)-\text{Var}(\bar X_n)\\&=\frac{1}{n}\text{Var}(X_i)-\text{Var}(\bar X_n)\\&=\frac{\sigma^2}{n}-\frac{\sigma^2}{n}=0\end{align}

Now you have to prove that $(\bar X_n,X_j-\bar X_n)$ is jointly normal for all $j=1,2,\cdots,n$ using MGF or otherwise. Once you have proved the joint normality, then the fact that $\bar X_n$ and $X_j-\bar X_n$ are uncorrelated would imply their independence. That is, $\bar X_n$ is independent of $X_1-\bar X_n,X_2-\bar X_n,\cdots,X_n-\bar X_n$, and hence also independent of $S_n^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar X_n)^2$.

To show the joint normality of $\bar X_n$ and $X_j-\bar X_n$, note that both $\bar X_n$ and $X_j-\bar X_n$ are linear combinations of independent normal variables $X_1,X_2,\cdots,X_n$ for all $j=1,2,\cdots,n$. As such, their joint distribution $(\bar X_n,X_j-\bar X_n)$ has to be bivariate normal.

For a formal proof, you may find the joint moment generating function (MGF) of, say, $(\bar X_n,X_1-\bar X_n)$ and show that the MGF is the MGF of a bivariate normal distribution.The details using MGF might get complicated, but to use the zero covariance proved in the first part to finally prove the independence of $(\bar X_n,S_n^2)$ you would have to show the joint normality somehow.

• Thanks a lot! I can totally see your conclusion with $0$ now. I propose to replace $\operatorname{Var}(X_i)$ with $\operatorname{Var}(X_j)$ for comprehensibility when $i \neq j$ is introduced. I will work on the jointly normal proof now. I am not familiar with MGFs though. – meisterluk Jun 19 '18 at 8:32
• @meisterluk Moment generating function (MGF) is not required to conclude joint normality. On the other hand, one can conclude independence as well as joint normality using only MGF. Showing the covariance vanishes would not be necessary in that case. – StubbornAtom Jun 19 '18 at 9:27
• @meisterluk To clarify, one can use MGF to directly conclude that $\bar X_n$ is independent of $(X_1-\bar X_n,\cdots,X_n-\bar X_n)$. But this is not what your exercise wants you to show. – StubbornAtom Jun 19 '18 at 9:40
• thx! As far as I can see joint normality follows from "the sum of normal distributions is a normal distribution". As far as $\rho(\overline{X_n}, X_j - \overline{X_n}) = 0$, because $\operatorname{Cov}(\overline{X_n}, X_j - \overline{X_n}) = 0$, independence follows. Did I get your hints right? – meisterluk Jun 19 '18 at 9:58
• Oh no, I forgot, that empirical variance employs a square. How can I show that $(X_j - \overline{X}_n)^2$ gives a normal distribution? – meisterluk Jun 19 '18 at 11:05