$X_1,X_2,\dots,X_n$ is an i.i.d. sequence of standard Gaussian random variables.

\begin{align}X&=\frac{1}{n}(X_1+X_2+\dots+X_n) \\[0.2cm] Y&=(X_1-X)^2+(X_2-X)^2+\dots+(X_n-X)^2\end{align}

  1. How can I show that $X$ and $Y$ are independent?
  2. How can I find the distribution of $Y$?

Can we use the following method to show that $X$ and $Y$ are independent? $$Cov(X,Y)=0$$

Or is there any other proper way?

  • 1
    $\begingroup$ In general, the covariance being zero will show that the two r.v.'s are uncorrelated, which does not mean they are independent. You have "independent implies uncorrelated," but the converse is false. $\endgroup$ – Clement C. Feb 2 '16 at 14:48
  • $\begingroup$ Thanks! Is there a way to do my questions? $\endgroup$ – Oily Feb 2 '16 at 14:50
  • $\begingroup$ A way, most likely... An elegant, way, maybe... one that I see right now, unfortunately no. $\endgroup$ – Clement C. Feb 2 '16 at 14:55
  • $\begingroup$ If you can show that for any two real numbers $u$ and $v$ we have $E[e^{uX+vY}]=E[e^{uX}]E[e^{vY}]$ then independence follows from Kac's theorem. $\endgroup$ – Elsa Feb 2 '16 at 15:04

The variable $X$ is the sample mean and the variable $Y$ is the sample variance times $(n-1)$. So Basu's theorem implies that they are independent.

The distribution of $Y$ is $\chi^2_{n-1}$ as the sum of the squares of the $n$ iid normal random variables $X_i-X$, (where $X$ is used and so there are $n-1$ degrees of freedom instead of $n$).


${\bf X}=(X_1,\dots, X_n)^\prime$ has a multivariate normal distribution with $\mu_{\bf X}=\mu {\bf 1}$ and $\Sigma_{\bf X}=\sigma^2 I$. Here ${\bf 1}$ is the column vector of all $1$s, while $I$ is the $n\times n$ identity matrix.

Let ${\bf e}_1=(1,0,0,\dots,0)^\prime $, and let $A$ be the matrix of an orthogonal transformation that takes the vector $\bf 1$ into the vector $\sqrt{n}\, {\bf e}_1$.

The vector ${\bf U}=A{\bf X}$ is multivariate normal with $\mu_{\bf U}=\mu \sqrt{n}\, {\bf e}_1 $ and $\Sigma_{\bf U}=\sigma^2 I$. In particular, the random variables $U_1,U_2,\dots, U_n$ are independent.

The first coordinate of the random vector $\bf U$ is $$U_1=(A{\bf X})^\prime{\bf e}_1={\bf X}^\prime A^\prime {\bf e}_1= {1\over \sqrt{n}}\, {\bf X}^\prime A^\prime A{\bf 1} = {1\over \sqrt{n}}\, {\bf X}^\prime {\bf 1}=\sqrt{n}\,X.$$

Also, $$\sum_{i=1}^n X^2_i={\bf X}^\prime {\bf X}= {\bf X}^\prime A^\prime A {\bf X}={\bf U}^\prime{\bf U} =n X^2+\sum_{i=2}^n U_i^2,$$ so that $$\sum_{i=1}^n (X_i-X)^2 =\sum_{i=1}^n X^2_i-nX^2=\sum_{i=2}^n U_i^2.$$

The independence of $U_1$ and $\sum_{i=2}^n U_i^2$ implies the independence of $X$ and $\sum_{i=1}^n (X_i-X)^2$.


Partial solution here.

$Y$ is a quadratic form. In particular, let $\mathbf{1} \in \mathbb{R}^n$ be a vector of ones, and define $$P_\mathbf{1} = \mathbf{1}\left(\mathbf{1}^{T}\mathbf{1}\right)^{-1}\mathbf{1}^{T} = \mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T}\text{.}$$ Let $$\mathbf{X}=\begin{bmatrix} X_1 \\ X_2 \\ \vdots\\ X_n \end{bmatrix}\text{.}$$ Then $$P_\mathbf{1}\mathbf{X} = \mathbf{1}\left(\dfrac{1}{n}\right)\begin{bmatrix} 1 & 1 & \cdots & 1 \end{bmatrix}\begin{bmatrix} X_1 \\ X_2 \\ \vdots\\ X_n \end{bmatrix} = \mathbf{1}\left(\dfrac{1}{n}\right)\sum_{i=1}^{n}X_i = \mathbf{1}X = X\mathbf{1}\text{.}$$ Furthermore, $$\begin{align} Y &= \sum_{i=1}^{n}(X_i-X)^2 \\ &= (\mathbf{X}-X\mathbf{1})^{T}(\mathbf{X}-X\mathbf{1}) \\ &= (\mathbf{X}-P_\mathbf{1}\mathbf{X})^{T}(\mathbf{x}-P_\mathbf{1}\mathbf{X}) \\ &= [(\mathbf{I}-P_{\mathbf{1}})\mathbf{X}]^{T}(\mathbf{I}-P_{\mathbf{1}})\mathbf{X} \\ &= \mathbf{X}^{T}(\mathbf{I}-P_{\mathbf{1}})^{T}(\mathbf{I}-P_{\mathbf{1}})\mathbf{X}\text{.} \end{align}$$ $\mathbf{I}$ is obviously symmetric, and notice $$P_{\mathbf{1}}^{T} = \left[\mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T}\right]^{T} = \mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T} = P_{\mathbf{1}}$$ so hence $P_{\mathbf{1}}$ is symmetric, so that $\mathbf{I}-P_{\mathbf{1}}$ is symmetric as well, and $$(\mathbf{I}-P_{\mathbf{1}})^{T} = \mathbf{I}-P_{\mathbf{1}}\text{.}$$ This gives $Y = \mathbf{X}^{T}(\mathbf{I}-P_{\mathbf{1}})^{2}\mathbf{X}$. Observe also that $$P_\mathbf{1}^2 = \left(\mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T}\right)^2 = \mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T}\mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T} = \mathbf{1}\left(\dfrac{n}{n}\right)\left(\dfrac{1}{n}\right)\mathbf{1}^{T} = \mathbf{1}\left(\dfrac{1}{n}\right)\mathbf{1}^{T} = P_{\mathbf{1}}$$ hence $P_{\mathbf{1}}$ is idempotent. Notice $$(\mathbf{I}-P_{\mathbf{1}})^{2} = \mathbf{I}^2-2\mathbf{I}P_{\mathbf{1}}+P_{\mathbf{1}}^2 = \mathbf{I}-2P_{\mathbf{1}}+P_{\mathbf{1}} = \mathbf{I}-P_{\mathbf{1}}$$ since $P_{\mathbf{1}}^2 = P_{\mathbf{1}}$, as shown earlier. Hence $Y = \mathbf{X}^{T}(\mathbf{I}-P_{\mathbf{1}})\mathbf{X}$.

It can be shown that the rank of $\mathbf{I}-P_{\mathbf{1}}$ is $n - 1$. Using the theorem 7 here (p. 29), you can show that $$Y \sim \sigma^2\chi^2_{n-1}(\boldsymbol{\mu}^{\prime}(\mathbf{I}-P_{\mathbf{1}})\boldsymbol{\mu}^{\prime})$$ If $$Q = \left(\mathbf{1}^{T}\mathbf{1}\right)^{-1}\mathbf{1}^{T} = \left(\dfrac{1}{n}\right)\mathbf{1}^{T}\text{.}$$ then $X = Q\mathbf{X}$.

For the remainder of the proof, see this page, under "Examples."


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.