Interested in meta-analysis I found the lecture notes http://www.uazuay.edu.ec/cibse/lecture_notes.pdf

I understand that it makes sense in a meta-analysis to assign different weights to different studies. Personally I find it natural to weigh by the sample sizes of the particular studies. However, in the fixed effect model the inverse of the variances is used. As an explanation on page 17 I read the sentence "The inverse variance is roughly proportional to sample size, but has finer distinctions,( and serves to minimize the variance of the combined effect.)"

I do not understand that and have problems with the first claim.

Suppose $X_1,..,X_n$ are random variables iid. Then $\bar{X}=\frac{1}{n} \sum_n X_i$ has variance $\frac{Var(X)}{n}$. This is clear. So one could say that on the level of random variables the inverse variance of the arithmetic mean is proportional to the sample size.

However, on the level of samples, we always compute the sample means and the sample variances, and the sample variance is an estimator for the variance of an individual measurement and not the variance of the (sample) arithmetic mean.

Can somebody clear up my confusion

  • $\begingroup$ So divide $s^2$ by $n$. And if studies $A$ and $B$ each weigh $100$ boxes of corn flakes, $A$ with a professional scale, $B$ with a dime store scale that has greater built in variance, in merging the results it makes sense to give greater weight to results from $A$. $\endgroup$ – André Nicolas Jul 11 '11 at 22:53
  • $\begingroup$ The first sentence above was a little abrupt. Indeed $s^2$ is an estimator for the population variance. Thus $s^2/n$ is an estimator for the variance of the sample mean. It should be good enough for determining appropriate weights when we do the pooling. $\endgroup$ – André Nicolas Jul 12 '11 at 6:00
  • $\begingroup$ That is exactly my problem. The weights come from consider the inverse of $s^2$, i.e. the estimator for the population variance and not for the estimator of the variance of the sample mean. $1/s^2$ will not be proportional to $n$, right? $\endgroup$ – wood Jul 12 '11 at 9:58
  • $\begingroup$ To be more precise. $s^2$ will be an estimator for the population variance. So the sample variance from two studies will approximately be the population variance. So we are weighing by roughly the same number, which does not make sense. Or is something else meant by weighing with variance? i.e. maybe not taking the sample variance but some other quantity? $\endgroup$ – wood Jul 12 '11 at 10:30
  • $\begingroup$ They probably mean weighting by inverse of $s^2/n$. Or jointly as inverse of $s^2$ and inverse of $n$. Only thing that is consistent with the quoted words. $\endgroup$ – André Nicolas Jul 12 '11 at 11:18

The notes you link to contain the answer (p. 16):

If all the studies included in the aggregation process were equally precise, it would suffice to estimate the effect size for each individual study and then average them out to get an overall effect size. In practice, though, not all the studies are equally precise. When combined, studies that supply more reliable information should be given more weight. A weighted mean is used to combine the results. [...]

As mentioned above, the overall effect is estimated as the weighted mean of the individual effects [Borenstein, M.; et al; 2007] [Hedges, L.; Olkin, I.; 1985], where each study is weighted as a function of the inverse variance. This way the more precise studies (generally studies including more experimental subjects) will be weighted higher because their results are considered to be more reliable or less error prone.

Thus, it is acknowledged that higher precision will generally be due to larger sample size (as you describe), but there are also other factors that may make one study more precise than another (such as the one user6312 mentioned in the comments), so weighting by the inverse variance is more general than weighting with the sample size. If all studies are otherwise equal and only differ in the sample sizes, the two approaches will yield the same result.

  • $\begingroup$ I understand that taking the variance may lead to a better result because it minimizes the variance of the combined effects. However, my question is what is mathematically meant by the inverse variance is proportional to the sample size. $\endgroup$ – wood Jul 12 '11 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.