# Proof that $\frac{(\bar X-\mu)}{\sigma}$ and $\sum_{i=1}^n\frac{(X_i-\bar X)^2}{\sigma^2}$ are independent

Let $X_i\sim N(\mu,\sigma^2)$ ; where$[i=1,2,\ldots,n]$

$Z_i\sim N(0,1)$ ; where$[i=1,2,\ldots,n]$

Proof that $\bar Z=\frac{(\bar X-\mu)}{\sigma}$ and $\sum_{i=1}^{n}(Z_i-\bar Z)^2=\sum_{i=1}^n\frac{(X_i-\bar X)^2}{\sigma^2}$ are independent, which implies $\bar X$ and $\sum_{i=1}^n(X_i-\bar X)^2$ are independent.

MY ATTEMPT:

I considered $n=2$, and

$$M_{X_1+X_2}(t_1)=M_{X_1}(t_1)M_{X_2}(t_1)$$

$\ast$I did so for the proof but does the statement $X_i\sim N(\mu,\sigma^2)$ ; where$[i=1,2,\ldots,n]$ imply that $X_i's$ are independent?

Moment Generating Function of $X_1+X_2$ and $X_1-X_2$ are $$M_{X_1+X_2}(t_1)=e^{2\mu t_1+\sigma^2 t_1^2}$$ $$M_{X_1-X_2}(t_2)=e^{\sigma^2 t_2^2}$$ respectively.

Also,

$$M_{X_1+X_2,X_1-X_2}(t_1,t_2)=e^{2\mu t_1+\sigma^2 t_1^2}e^{\sigma^2 t_2^2}=M_{X_1+X_2}(t_1)M_{X_1-X_2}(t_2)$$

since the joint moment generating function factors into the product of the marginal moment generating functions, so $X_1+X_2$ and $X_1-X_2$ are independent which implies that:

$\bullet$ Since $\bar Z=\frac{(\bar X-\mu)}{\sigma}$ is only a function of $X_1+X_2$ and $\sum_{i=1}^{2}(Z_i-\bar Z)^2=\sum_{i=1}^2\frac{(X_i-\bar X)^2}{\sigma^2}$ is only a function of $X_1-X_2$ ,so

$\bar Z=\frac{(\bar X-\mu)}{\sigma}$ and $\sum_{i=1}^{n}(Z_i-\bar Z)^2=\sum_{i=1}^n\frac{(X_i-\bar X)^2}{\sigma^2}$ are independent.

$\bullet$ Since $\bar X$ is only a function of $X_1+X_2$ and $S^2=\frac{1}{2-1}\sum_{i=1}^2(X_i-\bar X)^2$ is only a function of $X_1-X_2$ ,so

Sample mean,$\bar X$ and sample variance,$S^2$ are independent.

$\diamond\diamond\diamond$ We accept the above independence for any arbitrary $n$.

Is the procedure correct?

• I always check the independence of random variables by the uniqueness property of Moment Generating Function... And in this, quite misled you are: the factorization $M_{X+Y}(t)=M_X(t)M_Y(t)$ DOES NOT imply that $X$ and $Y$ are independent. (À propos, what happened to the MLE question I answered?)
– Did
Jun 22, 2013 at 9:21
• What are you talking about? The question was not edited since I answered it.
– Did
Jun 22, 2013 at 15:46
• (Some comments by the OP now erased.)
– Did
Jun 29, 2013 at 20:09
• Sep 17, 2020 at 8:16

The way to do this is to show that $\bar X=(X_1+\cdots+X_n)/n$ is independent of $$(X_1-\bar X, \ldots, X_n-\bar X).$$ Notice that the orthogonal projection of $(X_1,\ldots,X_n)$ onto the $1$-dimensional space $\{ (x,x,x,\ldots,x) : a \in \mathbb R\}$ is $(\bar X,\ldots, \bar X)$, and the orthogonal projection onto the $(n-1)$-dimensional space $x_1+\cdots+x_n=0$ is $(X_1-\bar X, \ldots, X_n-\bar X)$, and those two spaces are orthogonal to each other. Notice also that the probability distribution of $(X_1,\ldots,X_n)$ is spherically symmetric about the point $(\mu,\mu,\ldots,\mu)$, which is in the $1$-dimensional space mentioned above, and which gets mapped to $(0,0,0,\ldots,0)$ by the second projection.

Another way to look at it is this: The random vector $(X_1,\ldots,X_n)$ is jointly normally distributed, i.e. it is so distributed that every linear combination $a_1X_1+\cdots+a_nX_n$ has a $1$-dimensional normal distribution. That implies that any two linear combinations of $X_1,\ldots,X_n$ that are uncorrelated are independent. So then notice that $$\operatorname{cov}\left( \bar X, \begin{bmatrix} X_1 - \bar X \\ \vdots \\ X_n - \bar X\end{bmatrix}\right) = \begin{bmatrix} 0 \\ \vdots \\ 0 \end{bmatrix}.$$

• I am very weak in linear algebra.If i show that $\bar X$ and $X_i-\bar X$ are independent and draw conclusion that since $\bar Z$ is only the function of $\bar X$ and $\sum(Z_i-\bar Z)^2$ is only the function of $X_i-\bar X$, so $\bar Z$ and $\sum(Z_i-\bar Z)^2$ are independent. Is it suffices, ie, is the procedure to check the independence correct?
– time
Jun 22, 2013 at 13:46
• That should do it. Find $\operatorname{cov}(\bar X, X_i-\bar X)$. Then cite "joint" normality to explain why the fact that they're uncorrelated justifies saying they're independent. Jun 22, 2013 at 19:19
• I have edited the question. Could you please check it.
– time
Jun 23, 2013 at 3:51

One could use Cochran's theorem to prove this. Let $$J$$ be the $$n\times n$$ matrix of all ones. WLOG, $$X\sim N(0,I_n)$$. If $$Y\sim N(\mu,\sigma^2 I_n)$$, then we can standardize to get $$(Y-\mu)/\sigma\sim N(0,I_n)$$.

$$\sum_{i=1}^n X_i^2 = \sum_{i=1}^n (X_i - \bar{X})^2 + \frac{ (\sum_{i=1}^n X_i)^2 }{n} \\ \Rightarrow X'I_n X = X'(I_n - \frac{1}{n}J)X + X'(\frac{1}{n}J)X.$$

$$I_n - \frac{1}{n}J$$ has rank $$n-1$$, $$\frac{1}{n}J$$ has rank $$1$$, and $$I_n$$ has rank $$n$$, and they’re all PSD, thus the required conditions for Cochran's are satisfied. Thus $$\sum_{i=1}^n (X_i - \bar{X})^2$$ and $$\frac{ (\sum_{i=1}^n X_i)^2 }{n}$$ are independent and immediately, so are $$\frac{\sum_{i=1}^n (X_i - \bar{X})^2}{n-1}$$ and $$\frac{\sum_{i=1}^n X_i}{n}=\bar{X}$$ and we're done.

The last statement is because, for functions $$f$$ and $$g$$, $$f(X)$$ and $$g(Y)$$ are independent if $$X$$ and $$Y$$ are independent.

Also note that $$\sum_{i=1}^n (X_i - \bar{X})^2\sim \chi^2(n-1)$$ and $$\frac{ (\sum_{i=1}^n X_i)^2 }{n}\sim \chi^2(1)$$ from Cochran's as well.

================================================================== EDIT

Coming to think about this more, I have my doubts that we can use the above because I can't think of a univariate function that sends $$\frac{ (\sum X_i) ^2 }{ n}$$ to $$\sum X_i /n$$. $$\sqrt{ \frac{ (\sum X_i) ^2}{n}} * \frac{1}{\sqrt{n}} = \frac{\left| \sum X_i \right|}{n} \neq \frac{\sum X_i}{n} = \bar{X}.$$ However, I see the same reasoning as that of my answer in the literature. I just don't think it's correct.

For example, you could have a case where $$Y$$ and $$X^2$$ were independent but $$Y$$ and $$X$$ are not.

Let $$Y=2X$$ and $$X=\pm 1$$ with equal probabilities. Then $$X^2=1$$ a.s. and independent of $$Y$$. However, $$Y$$ depends on $$X$$.

So if someone can tell me why the Cochran's method would still work, that'd be helpful.

Instead, I'll prove the independence of sample mean and variance in a different way. I use a theorem about linear and quadratic forms that says $$AX$$ and $$X'BX$$ are independent if $$AB=0$$, assuming $$X\sim N(0,I_n)$$ and $$B$$ is symmetric and idempotent. Here, let $$A=(1/n,\ldots,1/n)^{\top}$$ and $$B=I_n - \frac{1}{n}J$$. Then it's easy to see the conditions are satisfied and $$X^{\top}(I_n - \frac{1}{n}J)X/(n-1)=s^2$$ is independent of $$(1/n,\ldots,1/n)^{\top} X=\bar{X}$$.

• Coming to think of this further, I have my doubts that we can use this because I can't think of a univariate function that sends $\frac{ (\sum X_i) ^2 }{ n}$ to $\sum X_i /n$. $$\sqrt{ \frac{ (\sum X_i) ^2}{n}} * \frac{1}{\sqrt{n}} = \left| \sum X_i \right| /n$$, which isn't what we want. Jun 19, 2018 at 5:03

There is a one-line proof, which, however, uses rather involved notions.

Namely, assuming $\sigma^2$ to be a known parameter, $\overline X$ is a boundedly complete sufficient statistic, while $\sum_{i=1}^n \frac{(X_i-\overline X)^2}{\sigma^2} \simeq \chi^2_{n-1}$ is an anciliary statistic. Therefore, by Basu's theorem, they are independent, and so are $\frac{\overline X - \mu}{\sigma}$ and $\sum_{i=1}^n \frac{(X_i-\overline X)^2}{\sigma^2}$.