4
$\begingroup$

I'm trying to see the relationship between the sample variance equation

$\sum(X_i- \bar X)^2/(n-1)$ and the variance estimate, $\bar X(1-\bar X),$ in case of binary samples.

I wonder if the outputs are the same, or if not, what is the relationship between the two??

I'm trying to prove their relationship but it's quite challenging to me..

Please help!

Sigma(Xi-Xbar)/(n-1) Xbar(1-Xbar)

$\endgroup$
  • $\begingroup$ I edited your question. Please make sure I did not change what you intended. $\endgroup$ – BruceET Dec 14 '15 at 2:27
  • $\begingroup$ They are not the same. Try $n=2$, $X_1=0, X_2=1$. $\endgroup$ – A.S. Dec 14 '15 at 2:40
  • 1
    $\begingroup$ The latter is the former times $\frac {n-1}n$. So the latter is just the usual, naive biased estimator of the variance. $\endgroup$ – A.S. Dec 14 '15 at 2:44
  • $\begingroup$ I've posted a straightforward answer where the others don't get right to the point. $\qquad$ $\endgroup$ – Michael Hardy Jul 18 '16 at 3:17
2
$\begingroup$

I suppose your question is whether the two formulas give the same answer for binary data. Here is an example to illustrate that they are almost the same, but not exactly.

Suppose I have a sample of a thousand zeros and ones in which there are 283 ones. Then $\bar X = 283/1000 = 0.283.$ Thus, $\bar X(1-\bar X) = 0.283(1 - 0.283) = 0.202911.$

An alternate general formula for the sample variance of values $X_i$ is

$$S^2 = \frac{\sum_{i=1}^n X_i^2 - n \bar X^2}{n-1}.$$

In a binary sample $\sum_{i=1}^n X_i^2 = \sum_{i=1}^n X_i$, because $0^2 = 0$ and $1^2 = 1.$

Thus, the general formula gives $S^2 = \frac{283 - 1000(.283)^2}{999} = 0.2031141.$ If (as in the Comment by @A.S) the denominator were $n = 1000$ instead of $n-1=999,$ this would simplify to $$S^2 = 0.283 - 0.283^2 = 0.283(1 = 0.283) = \bar X(1- \bar X).$$

The formula for the population variance is often written with the population size $n$ in the denominator.

$\endgroup$
1
$\begingroup$

The first quantity is the standard variance estimator that is unbiased for i.i.d samples from any distribution.

The second quantity is a simplified formula (the simplification being valid only for 0-1 binary data) for calculating exactly, not estimating, the variance of the sample.

Using the second instead of the first to estimate the distribution variance will, on average, lead to slight underestimates. This is equivalent to the use of $n$ instead of $n-1$ in the denominator of the estimator.

$\endgroup$
1
$\begingroup$

\begin{align} & \sum_{i=1}^n (x_i - \bar x)^2 \\[10pt] = {} & \sum_{i=1}^n (x_i - 2\bar x x_i + \bar x^2) \\[10pt] = {} & \left( \sum_{i=1}^n x_i^2 \right) - 2\bar x \left( \sum_{i=1}^n x_i \right) + \left( \sum_{i=1}^n \bar x^2 \right) \\[10pt] = {} & \left( \sum_{i=1}^n x_i^2 \right) - 2\bar x\Big( n\bar x\Big) + n \bar x^2 \\[10pt] = {} & \left( \sum_{i=1}^n x_i^2 \right) - n \bar x^2 \\[10pt] = {} & \left( \sum_{i=1}^n x_i \right) - n \bar x^2 \qquad \text{since } x_i \in \{0,1\} \\[10pt] = {} & n\bar x - n \bar x^2 = n\bar x(1-\bar x). \end{align} $$ \text{So } \qquad \frac 1 n \sum_{i=1}^n (x_i - \bar x)^2 = \bar x (1 - \bar x). $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.