5
$\begingroup$

I have seen the text book derivation where the independence is established through factoring the joint distribution. But has anyone tried to prove that the covariance is zero?.

Let $Z_{i}$ come from a standard normal distribution.

Let $X = \bar{Z}$ be the sample mean and let $$Y = \sum_i\frac{1}{n-1} (Z_{i} -\bar{Z})^2$$ be the sample variance.

Prove that $\operatorname{Cov}(X,Y) = 0$ .

I am getting terms like E($\bar{Z}^3$) in the expansion which is making it very cumbersome to handle...

$\endgroup$
6
  • $\begingroup$ By symmetry of an odd function, $E\left[\bar{Z}^3\right] = 0$ for a standard normal distribution. $\endgroup$
    – Henry
    Feb 19, 2012 at 21:49
  • $\begingroup$ It happens that the simplest (and most illuminating) way to prove that $X$ and $Y$ are uncorrelated is to show that they are independent... $\endgroup$
    – Did
    Feb 19, 2012 at 22:12
  • $\begingroup$ ^ yes I understand ... but it turned out to be very convenient that the JDF/MGF was obtained and could be factored for the normal dist $\endgroup$
    – dasman
    Feb 19, 2012 at 22:48
  • 2
    $\begingroup$ The title of your question "Independence of sample mean and variance through covariance" concerns me a bit. Are you trying to establish that $X$ and $Y$ are independent by showing that their covariance is zero? $\endgroup$
    – Dilip Sarwate
    Feb 20, 2012 at 3:32
  • $\begingroup$ ^ I was... but I realize thats a wrong approach. $\endgroup$
    – dasman
    Feb 20, 2012 at 5:35

2 Answers 2

Reset to default
6
$\begingroup$

Consider the random vectors $Z=(Z_i)_{1\leqslant i\leqslant n}$ and $-Z=(-Z_i)_{1\leqslant i\leqslant n}$. Then $X=\xi(Z)$ for a given odd function $\xi$ and $Y=\eta(Z)$ for a given even function $\eta$. The function $\zeta=\xi\cdot\eta$ is odd as well, hence $\xi(-Z)=-\xi(Z)$ and $\zeta(-Z)=-\zeta(Z)$. Now, $Z$ and $-Z$ follow the same distribution, hence $\xi$ and $\zeta$ being odd functions yields that $\mathrm E(\xi(Z))=\mathrm E(\zeta(Z))=0$. In particular, the covariance of $X$ and $Y$ is $\mathrm E(\zeta(Z))-\mathrm E(\xi(Z))\mathrm E(\eta(Z))=0$.

To sum up, the result you ask a simple proof of (that the empirical mean and empirical variance are uncorrelated) has nothing to do with gaussianity since it holds for every symmetric distribution.

$\endgroup$
2
  • $\begingroup$ Thanks for the detailed explanation... I understand what you say now but I accepted Michael's answer as it was more direct $\endgroup$
    – dasman
    Feb 20, 2012 at 5:50
  • 2
    $\begingroup$ +1, especially because it is not necessary to assume the $Z_i$ are independent or uncorrelated, but I have a suggestion and a minor question. Suggestion: it is not obvious why $Z$ and $-Z$ follow the same distribution, and so perhaps the statement about symmetric distribution should be moved to the beginning. Minor question: The expectation of an odd function of a random variable might be undefined rather than being $0$. So, should the proof be expanded to show that for these particular odd functions the expectations are zero and so the empirical mean and variance are uncorrelated? $\endgroup$
    – Dilip Sarwate
    Feb 20, 2012 at 14:28
4
$\begingroup$

First show that $$ \operatorname{cov} (\bar{Z}, Z_i - \bar{Z}) = 0. $$ These two random variables are not just normally distributed, but jointly normally distributed, since both are linear combinations of the same sequence of independent normals. Therefore, they are uncorrelated only if they're independent.

Hence the sample variance, which is a function of $Z_i-\bar{Z}$ for $i=1,\ldots,n$, is independent of the sample mean.

Later addendum: I should add that the whole vector $(Z_1- \bar Z,\ldots,Z_n-\bar Z)$ is independent of $\bar Z$, since the covariance between $\bar Z$ and that vector is a matrix whose every entry is $0$ and we have joint normality. If $(Z_1- \bar Z,\ldots,Z_n-\bar Z)$ is independent of $\bar Z$, then $((Z_1- \bar Z)^2+\cdots+(Z_n-\bar Z)^2$ is independent of $\bar Z$.

$\endgroup$
6
  • $\begingroup$ Sorry but $\bar Z$ and $Z_i-\bar Z$ being independent for every $i$ (as you recall) does NOT imply that a function of $(Z_i-\bar Z)_i$ (such as $Y$) is independent on $\bar Z$. $\endgroup$
    – Did
    Feb 21, 2012 at 6:55
  • $\begingroup$ I've added a comment that it's the whole vector whose entries are $Z_i - \bar Z$ that is independent of $\bar Z$. That is enough to justify the conclusion. $\endgroup$
    – Michael Hardy
    Feb 23, 2012 at 18:24
  • 1
    $\begingroup$ Why vector $(Z_{1}-\bar{Z} \dots Z_{n} - \bar{Z})$ and $\bar{Z}$ are independent ? $\endgroup$
    – openspace
    Sep 18, 2018 at 10:10
  • $\begingroup$ @openspace : First note that the covariance between $\overline Z$ and $Z_i-\overline Z$ is $0.$ Then observe that $\big(Z_1-\overline Z, \ldots, Z_n-\overline Z, \overline Z\,\big)$ is a tuple of JOINTLY normally distributed random variables. $\qquad$ $\endgroup$
    – Michael Hardy
    Sep 18, 2018 at 13:12
  • $\begingroup$ @MichaelHardy jointly because of their independence? $\endgroup$
    – openspace
    Sep 18, 2018 at 13:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .