Let $s_p^2 = bs_1^2 + (1-b)s_2^2$, this can be an unbiased estimator of population variance, provided we find the correct value for $b$; in particular, $s_p^2 = \frac{(n1-1)s_1^2 + (n_2-1)s_2^2}{n_1+n_2-2}$ is an unbiased estimator

To minimize variance of $s_p^2 = bs_1^2 + (1-b)s_2^2:V(s_p^2)=b^2 V(s_1^2) + (1-b)^2 V(s_2^2)$ we need to find $\frac{d}{db}V(s_p^2)=0$ but I cannot solve this for $s_p^2$.

Intuitively, would think that the weighted average minimizes the variance; however, I want to know the precise proof of this.

How can i get this?

Thank you in advance.


Lets start with $s_p^2 = bs_1^2 + (1-b)s_2^2:V(s_p^2)=b^2 V(s_1^2) + (1-b)^2 V(s_2^2)$

$\frac{d}{db} V(s_p^2)=2bV(s_1^2)-2(1-b)V(s_2^2)=0 \implies \frac{b}{1-b} = \frac{Var(s_2^2)}{Var(s_1^2)}$

As you can see, $b,(1-b)$ is inversely proportional to the variance of $s_1^2,s_2^2$ respectively.

Now, if they both come from the same population (with assumed same variance), this implies that $Var(s_1^2)=\frac{n_2}{n_1} Var(s_2^2) \rightarrow \frac{b}{1-b} = \frac{Var(s_2^2)}{Var(s_1^2)}=\frac{n_1}{n_2} \implies n_2b-n_1(1-b)=b(n_2+n_1)-n_1=0 \rightarrow b=\frac{n_1}{n_1+n_2}$

This means that the larger the sample size for sample 1 relative to sample 2, the more weight we give to sample 1.

| cite | improve this answer | |
  • $\begingroup$ Thank you! Finally I got it! $\endgroup$ – Choi Aug 25 '14 at 13:50
  • $\begingroup$ But "b" should be n1-1/(n1+n2-2) !? $\endgroup$ – Choi Aug 25 '14 at 14:08
  • $\begingroup$ @choi: sorry, you're right. I was using the maximum likelihood estimator of the variance. Just substitute $(n-1)$ for for $n$ and you should be OK. $\endgroup$ – user76844 Aug 25 '14 at 14:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.