2
$\begingroup$

Let $X_i\sim N(\mu,\sigma^2)$ ; where$[i=1,2,\ldots,n]$

$Z_i\sim N(0,1)$ ; where$[i=1,2,\ldots,n]$

Proof that $\bar Z=\frac{(\bar X-\mu)}{\sigma}$ and $\sum_{i=1}^{n}(Z_i-\bar Z)^2=\sum_{i=1}^n\frac{(X_i-\bar X)^2}{\sigma^2}$ are independent, which implies $\bar X$ and $\sum_{i=1}^n(X_i-\bar X)^2$ are independent.

MY ATTEMPT:

I considered $n=2$, and

$$M_{X_1+X_2}(t_1)=M_{X_1}(t_1)M_{X_2}(t_1)$$

$\ast$I did so for the proof but does the statement $X_i\sim N(\mu,\sigma^2)$ ; where$[i=1,2,\ldots,n]$ imply that $X_i's$ are independent?

Moment Generating Function of $X_1+X_2$ and $X_1-X_2$ are $$M_{X_1+X_2}(t_1)=e^{2\mu t_1+\sigma^2 t_1^2}$$ $$M_{X_1-X_2}(t_2)=e^{\sigma^2 t_2^2}$$ respectively.

Also,

$$M_{X_1+X_2,X_1-X_2}(t_1,t_2)=e^{2\mu t_1+\sigma^2 t_1^2}e^{\sigma^2 t_2^2}=M_{X_1+X_2}(t_1)M_{X_1-X_2}(t_2)$$

since the joint moment generating function factors into the product of the marginal moment generating functions, so $X_1+X_2$ and $X_1-X_2$ are independent which implies that:

$\bullet$ Since $\bar Z=\frac{(\bar X-\mu)}{\sigma}$ is only a function of $X_1+X_2$ and $\sum_{i=1}^{2}(Z_i-\bar Z)^2=\sum_{i=1}^2\frac{(X_i-\bar X)^2}{\sigma^2}$ is only a function of $X_1-X_2$ ,so

$\bar Z=\frac{(\bar X-\mu)}{\sigma}$ and $\sum_{i=1}^{n}(Z_i-\bar Z)^2=\sum_{i=1}^n\frac{(X_i-\bar X)^2}{\sigma^2}$ are independent.

$\bullet$ Since $\bar X$ is only a function of $X_1+X_2$ and $S^2=\frac{1}{2-1}\sum_{i=1}^2(X_i-\bar X)^2$ is only a function of $X_1-X_2$ ,so

Sample mean,$\bar X$ and sample variance,$S^2$ are independent.

$\diamond\diamond\diamond$ We accept the above independence for any arbitrary $n$.

Is the procedure correct?

$\endgroup$
  • $\begingroup$ I always check the independence of random variables by the uniqueness property of Moment Generating Function... And in this, quite misled you are: the factorization $M_{X+Y}(t)=M_X(t)M_Y(t)$ DOES NOT imply that $X$ and $Y$ are independent. (À propos, what happened to the MLE question I answered?) $\endgroup$ – Did Jun 22 '13 at 9:21
  • $\begingroup$ What are you talking about? The question was not edited since I answered it. $\endgroup$ – Did Jun 22 '13 at 15:46
  • $\begingroup$ (Some comments by the OP now erased.) $\endgroup$ – Did Jun 29 '13 at 20:09
3
$\begingroup$

One could use Cochran's theorem to prove this. Let $J$ be the $n\times n$ matrix of all ones. WLOG, $X\sim N(0,I_n)$. If $Y\sim N(\mu,\sigma^2 I_n)$, then we can standardize to get $(Y-\mu)/\sigma\sim N(0,I_n)$.

$$\sum_{i=1}^n X_i^2 = \sum_{i=1}^n (X_i - \bar{X})^2 + \frac{ (\sum_{i=1}^n X_i)^2 }{n} \\ \Rightarrow X'I_n X = X'(I_n - \frac{1}{n}J)X + X'(\frac{1}{n}J)X.$$

$I_n - \frac{1}{n}J$ has rank $n-1$, $\frac{1}{n}J$ has rank $1$, and $I_n$ has rank $n$, and they’re all PSD, thus the required conditions for Cochran's are satisfied. Thus $\sum_{i=1}^n (X_i - \bar{X})^2$ and $\frac{ (\sum_{i=1}^n X_i)^2 }{n}$ are independent and immediately, so are $\frac{\sum_{i=1}^n (X_i - \bar{X})^2}{n-1}$ and $\frac{\sum_{i=1}^n X_i}{n}=\bar{X}$ and we're done.

The last statement is because, for functions $f$ and $g$, $f(X)$ and $g(Y)$ are independent if $X$ and $Y$ are independent.

Also note that $\sum_{i=1}^n (X_i - \bar{X})^2\sim \chi^2(n-1)$ and $\frac{ (\sum_{i=1}^n X_i)^2 }{n}\sim \chi^2(1)$ from Cochran's as well.

================================================================== EDIT

Coming to think about this more, I have my doubts that we can use the above because I can't think of a univariate function that sends $\frac{ (\sum X_i) ^2 }{ n}$ to $\sum X_i /n$. $$\sqrt{ \frac{ (\sum X_i) ^2}{n}} * \frac{1}{\sqrt{n}} = \frac{\left| \sum X_i \right|}{n} \neq \frac{\sum X_i}{n} = \bar{X}.$$ However, I see the same reasoning as that of my answer in the literature. I just don't think it's correct.

For example, you could have a case where $Y$ and $X^2$ were independent but $Y$ and $X$ are not.

Let $Y=2X$ and $X=\pm 1$ with equal probabilities. Then $X^2=1$ a.s. and independent of $Y$. However, $Y$ depends on $X$.

So if someone can tell me why the Cochran's method would still work, that'd be helpful.

Instead, I'll prove the independence of sample mean and variance in a different way. I use a theorem about linear and quadratic forms that says $AX$ and $X'BX$ are independent if $AB=0$, assuming $X\sim N(0,I_n)$ and $B$ is symmetric and idempotent. Here, let $A=(1/n,\ldots,1/n)^{\top}$ and $B=I_n - \frac{1}{n}J$. Then it's easy to see the conditions are satisfied and $X^{\top}(I_n - \frac{1}{n}J)X/(n-1)=s^2$ is independent of $(1/n,\ldots,1/n)^{\top} X=\bar{X}$.

$\endgroup$
  • $\begingroup$ Coming to think of this further, I have my doubts that we can use this because I can't think of a univariate function that sends $\frac{ (\sum X_i) ^2 }{ n}$ to $\sum X_i /n$. $$\sqrt{ \frac{ (\sum X_i) ^2}{n}} * \frac{1}{\sqrt{n}} = \left| \sum X_i \right| /n$$, which isn't what we want. $\endgroup$ – Glassjawed Jun 19 '18 at 5:03
5
$\begingroup$

The way to do this is to show that $\bar X=(X_1+\cdots+X_n)/n$ is independent of $$ (X_1-\bar X, \ldots, X_n-\bar X). $$ Notice that the orthogonal projection of $(X_1,\ldots,X_n)$ onto the $1$-dimensional space $\{ (x,x,x,\ldots,x) : a \in \mathbb R\}$ is $(\bar X,\ldots, \bar X)$, and the orthogonal projection onto the $(n-1)$-dimensional space $x_1+\cdots+x_n=0$ is $(X_1-\bar X, \ldots, X_n-\bar X)$, and those two spaces are orthogonal to each other. Notice also that the probability distribution of $(X_1,\ldots,X_n)$ is spherically symmetric about the point $(\mu,\mu,\ldots,\mu)$, which is in the $1$-dimensional space mentioned above, and which gets mapped to $(0,0,0,\ldots,0)$ by the second projection.

Another way to look at it is this: The random vector $(X_1,\ldots,X_n)$ is jointly normally distributed, i.e. it is so distributed that every linear combination $a_1X_1+\cdots+a_nX_n$ has a $1$-dimensional normal distribution. That implies that any two linear combinations of $X_1,\ldots,X_n$ that are uncorrelated are independent. So then notice that $$ \operatorname{cov}\left( \bar X, \begin{bmatrix} X_1 - \bar X \\ \vdots \\ X_n - \bar X\end{bmatrix}\right) = \begin{bmatrix} 0 \\ \vdots \\ 0 \end{bmatrix}. $$

$\endgroup$
  • $\begingroup$ I am very weak in linear algebra.If i show that $\bar X$ and $X_i-\bar X$ are independent and draw conclusion that since $\bar Z$ is only the function of $\bar X$ and $\sum(Z_i-\bar Z)^2$ is only the function of $X_i-\bar X$, so $\bar Z$ and $\sum(Z_i-\bar Z)^2$ are independent. Is it suffices, ie, is the procedure to check the independence correct? $\endgroup$ – time Jun 22 '13 at 13:46
  • 1
    $\begingroup$ That should do it. Find $\operatorname{cov}(\bar X, X_i-\bar X)$. Then cite "joint" normality to explain why the fact that they're uncorrelated justifies saying they're independent. $\endgroup$ – Michael Hardy Jun 22 '13 at 19:19
  • $\begingroup$ I have edited the question. Could you please check it. $\endgroup$ – time Jun 23 '13 at 3:51
2
$\begingroup$

There is a one-line proof, which, however, uses rather involved notions.

Namely, assuming $\sigma^2$ to be a known parameter, $\overline X$ is a boundedly complete sufficient statistic, while $\sum_{i=1}^n \frac{(X_i-\overline X)^2}{\sigma^2} \simeq \chi^2_{n-1}$ is an anciliary statistic. Therefore, by Basu's theorem, they are independent, and so are $\frac{\overline X - \mu}{\sigma}$ and $\sum_{i=1}^n \frac{(X_i-\overline X)^2}{\sigma^2}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.