1
$\begingroup$

I'm trying to work through the proof given here that the sample mean and sample variance of a random sample $X_1, X_2, ..., X_n \sim N(\mu,\sigma^2)$ are independent.

The part I can't seem to follow is the assertion that the Jacobian of the transformations...

$$ Y_1 = \bar{X} \\ Y_2 = X_2 - \bar{X} \\ \vdots \\ Y_n = X_n - \bar{X} \\ $$

is equal to $n$. I must be missing something simple, but when I create the matrix of partial derivatives, the determinant seems to always be one?

$$ \det\begin{bmatrix} \frac{dY_1}{d\bar{X}} & \frac{dY_1}{dX_2} & \dots & \frac{dY_1}{dX_n} \\ \frac{dY_2}{d\bar{X}} & \frac{dY_2}{dX_2} & \dots & \frac{dY_2}{dX_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{dY_n}{d\bar{X}} & \frac{dY_n}{dX_2} & \dots & \frac{dY_n}{dX_n} \\ \end{bmatrix} = \det\begin{bmatrix} 1 & 0 & \dots & 0 \\ -1 & 1 & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ -1 & 0 & \dots & 1 \\ \end{bmatrix} = 1 $$

How am I setting up the Jacobian incorrectly? Thanks in advance!

$\endgroup$
  • 1
    $\begingroup$ the differentials have to be with respect to $X_1,X_2,X_3,....,X_n$. Hope you would get it now.... $\endgroup$ – mint Feb 7 '16 at 23:01
  • 1
    $\begingroup$ Thanks, substituting $\bar{X} = \frac{1}{n}\sum_{i=0}^nX_i$ creates a matrix with a determinant of $\frac{1}{n}$. $\endgroup$ – Ben Southgate Feb 7 '16 at 23:28
2
$\begingroup$

Comments: This result is true only for normal data.

The idea is to do an orthogonal transformation in which one of the new random variables is a function of $\bar X$ and the other $n-1$ are a function of $S^2$. For a multivariate normal, orthogonality is equivalent to independence.

You might start with $n = 2$ where the proof is especially simple. Let $Y_1 = X_1 + X_2$ and $Y_2 = X_1 - X_2.$ (A 45 degree rotation.) If $X_1$ and $X_2$ are independent, then so are $Y_1$ and $Y_2.$ It is obvious that $\bar X$ is a function of $Y_1,$ and not difficult to show that $S^2$ is a function of $Y_2$.

Illustration of independence of $\bar X$ and $S$ for normal, but not for nonnormal data: In each of the panels below 30,000 samples of size 5 are simulated, standard normal on the left and $Beta(.5, .5)$ on the right. In both cases, sample correlations are consistent with $\rho = 0$ (by symmetry). For normal data, $\bar X$ and $S$ are independent, and for beta data, they are obviously not. (The figure at right illustrates that a 5-D hypercube has been 'squashed' onto two dimensions. $Beta(.5,.5)$ puts most of its probability near corners, edges and (hyper)faces, images of some of these, after 'squashing', are discernible.)

enter image description here

In case you want it, the R code is shown below.

 par(mfrow=c(1,2))
  m = 30000;  n = 5
   DTA = matrix(rnorm(m*n), nrow=m)  # each row a sample of size n
   a = rowMeans(DTA);  s = apply(DTA, 1, sd)  # sample mean and sd
   cor(a,s)
   ## -0.0066071
   plot(a, s, pch=".", main="Normal: Mean vs SD")
  m = 30000;  n = 5
   DTA = matrix(rbeta(m*n, .5, .5), nrow=m)  # non-normal data
   a = rowMeans(DTA);  s = apply(DTA, 1, sd)  # sample mean and sd
   cor(a,s)
   ## -0.005415171
   plot(a, s, pch=".", main="BETA: Mean vs SD")
 par(mfrow=c(1,1))
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.