# How can I show that $X$ and $Y$ are independent and find the distribution of $Y$?

$X_1,X_2,\dots,X_n$ is an i.i.d. sequence of standard Gaussian random variables.

\begin{align}X&=\frac{1}{n}(X_1+X_2+\dots+X_n) \\[0.2cm] Y&=(X_1-X)^2+(X_2-X)^2+\dots+(X_n-X)^2\end{align}

1. How can I show that $X$ and $Y$ are independent?
2. How can I find the distribution of $Y$?

Can we use the following method to show that $X$ and $Y$ are independent? $$Cov(X,Y)=0$$

Or is there any other proper way?

• In general, the covariance being zero will show that the two r.v.'s are uncorrelated, which does not mean they are independent. You have "independent implies uncorrelated," but the converse is false. – Clement C. Feb 2 '16 at 14:48
• Thanks! Is there a way to do my questions? – Oily Feb 2 '16 at 14:50
• A way, most likely... An elegant, way, maybe... one that I see right now, unfortunately no. – Clement C. Feb 2 '16 at 14:55
• If you can show that for any two real numbers $u$ and $v$ we have $E[e^{uX+vY}]=E[e^{uX}]E[e^{vY}]$ then independence follows from Kac's theorem. – Elsa Feb 2 '16 at 15:04

## 3 Answers

The variable $X$ is the sample mean and the variable $Y$ is the sample variance times $(n-1)$. So Basu's theorem implies that they are independent.

The distribution of $Y$ is $\chi^2_{n-1}$ as the sum of the squares of the $n$ iid normal random variables $X_i-X$, (where $X$ is used and so there are $n-1$ degrees of freedom instead of $n$).

${\bf X}=(X_1,\dots, X_n)^\prime$ has a multivariate normal distribution with $\mu_{\bf X}=\mu {\bf 1}$ and $\Sigma_{\bf X}=\sigma^2 I$. Here ${\bf 1}$ is the column vector of all $1$s, while $I$ is the $n\times n$ identity matrix.

Let ${\bf e}_1=(1,0,0,\dots,0)^\prime$, and let $A$ be the matrix of an orthogonal transformation that takes the vector $\bf 1$ into the vector $\sqrt{n}\, {\bf e}_1$.

The vector ${\bf U}=A{\bf X}$ is multivariate normal with $\mu_{\bf U}=\mu \sqrt{n}\, {\bf e}_1$ and $\Sigma_{\bf U}=\sigma^2 I$. In particular, the random variables $U_1,U_2,\dots, U_n$ are independent.

The first coordinate of the random vector $\bf U$ is $$U_1=(A{\bf X})^\prime{\bf e}_1={\bf X}^\prime A^\prime {\bf e}_1= {1\over \sqrt{n}}\, {\bf X}^\prime A^\prime A{\bf 1} = {1\over \sqrt{n}}\, {\bf X}^\prime {\bf 1}=\sqrt{n}\,X.$$

Also, $$\sum_{i=1}^n X^2_i={\bf X}^\prime {\bf X}= {\bf X}^\prime A^\prime A {\bf X}={\bf U}^\prime{\bf U} =n X^2+\sum_{i=2}^n U_i^2,$$ so that $$\sum_{i=1}^n (X_i-X)^2 =\sum_{i=1}^n X^2_i-nX^2=\sum_{i=2}^n U_i^2.$$

The independence of $U_1$ and $\sum_{i=2}^n U_i^2$ implies the independence of $X$ and $\sum_{i=1}^n (X_i-X)^2$.

Partial solution here.

$Y$ is a quadratic form. In particular, let $\mathbf{1} \in \mathbb{R}^n$ be a vector of ones, and define $$P_\mathbf{1} = \mathbf{1}\left(\mathbf{1}^{T}\mathbf{1}\right)^{-1}\mathbf{1}^{T} = \mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T}\text{.}$$ Let $$\mathbf{X}=\begin{bmatrix} X_1 \\ X_2 \\ \vdots\\ X_n \end{bmatrix}\text{.}$$ Then $$P_\mathbf{1}\mathbf{X} = \mathbf{1}\left(\dfrac{1}{n}\right)\begin{bmatrix} 1 & 1 & \cdots & 1 \end{bmatrix}\begin{bmatrix} X_1 \\ X_2 \\ \vdots\\ X_n \end{bmatrix} = \mathbf{1}\left(\dfrac{1}{n}\right)\sum_{i=1}^{n}X_i = \mathbf{1}X = X\mathbf{1}\text{.}$$ Furthermore, \begin{align} Y &= \sum_{i=1}^{n}(X_i-X)^2 \\ &= (\mathbf{X}-X\mathbf{1})^{T}(\mathbf{X}-X\mathbf{1}) \\ &= (\mathbf{X}-P_\mathbf{1}\mathbf{X})^{T}(\mathbf{x}-P_\mathbf{1}\mathbf{X}) \\ &= [(\mathbf{I}-P_{\mathbf{1}})\mathbf{X}]^{T}(\mathbf{I}-P_{\mathbf{1}})\mathbf{X} \\ &= \mathbf{X}^{T}(\mathbf{I}-P_{\mathbf{1}})^{T}(\mathbf{I}-P_{\mathbf{1}})\mathbf{X}\text{.} \end{align} $\mathbf{I}$ is obviously symmetric, and notice $$P_{\mathbf{1}}^{T} = \left[\mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T}\right]^{T} = \mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T} = P_{\mathbf{1}}$$ so hence $P_{\mathbf{1}}$ is symmetric, so that $\mathbf{I}-P_{\mathbf{1}}$ is symmetric as well, and $$(\mathbf{I}-P_{\mathbf{1}})^{T} = \mathbf{I}-P_{\mathbf{1}}\text{.}$$ This gives $Y = \mathbf{X}^{T}(\mathbf{I}-P_{\mathbf{1}})^{2}\mathbf{X}$. Observe also that $$P_\mathbf{1}^2 = \left(\mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T}\right)^2 = \mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T}\mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T} = \mathbf{1}\left(\dfrac{n}{n}\right)\left(\dfrac{1}{n}\right)\mathbf{1}^{T} = \mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T} = P_{\mathbf{1}}$$ hence $P_{\mathbf{1}}$ is idempotent. Notice $$(\mathbf{I}-P_{\mathbf{1}})^{2} = \mathbf{I}^2-2\mathbf{I}P_{\mathbf{1}}+P_{\mathbf{1}}^2 = \mathbf{I}-2P_{\mathbf{1}}+P_{\mathbf{1}} = \mathbf{I}-P_{\mathbf{1}}$$ since $P_{\mathbf{1}}^2 = P_{\mathbf{1}}$, as shown earlier. Hence $Y = \mathbf{X}^{T}(\mathbf{I}-P_{\mathbf{1}})\mathbf{X}$.

It can be shown that the rank of $\mathbf{I}-P_{\mathbf{1}}$ is $n - 1$. Using the theorem 7 here (p. 29), you can show that $$Y \sim \sigma^2\chi^2_{n-1}(\boldsymbol{\mu}^{\prime}(\mathbf{I}-P_{\mathbf{1}})\boldsymbol{\mu}^{\prime})$$ If $$Q = \left(\mathbf{1}^{T}\mathbf{1}\right)^{-1}\mathbf{1}^{T} = \left(\dfrac{1}{n}\right)\mathbf{1}^{T}\text{.}$$ then $X = Q\mathbf{X}$.

For the remainder of the proof, see this page, under "Examples."