4
$\begingroup$

This may end up being a question more about scientific best practice than anything else, but I think this is the right community to ask it in to get the insight I'm looking for.


Say I have two little square widgets made out of a material that shrinks when it gets wet. I want to know by how much. I measure the length of the widgets along two lines each (because they're not shaped perfectly and my measurement technique isn't perfect), before and after soaking them with water. I come back with data that looks like this:

Widget  Measurement  Before  After  Shrinkage
1       1            1.898   1.722  0.176
1       2            1.904   1.737  0.167
2       1            2.003   1.763  0.240
2       2            2.029   1.843  0.186

Now, I can calculate the overall mean without worrying too much in this case, since the mean of two means is the same as the mean of all the points that went in as long as each mean has the same number of samples, which in this case they do. So:

avg(0.176,0.167,0.240,0.186) = 0.192 = avg(avg(0.176,0.167),avg(0.240,0.186))

However, this type of relation is not true for the standard deviation. There are several approaches that immediately present themselves to me as options for finding an overall standard deviation for this dataset:

  1. Use all of the data at once: sd(0.176,0.167,0.240,0.186) = 0.033
  2. Get a standard deviation for each widget, and average them: avg(sd(0.176,0.167),sd(0.240,0.186)) = 0.022
  3. Get the average for each widget, and take the standard deviation of the two: sd(avg(0.176,0.167),avg(0.240,0.186)) = 0.029

Now, maybe it's just confusion on my part as to the meaning of a standard deviation, but I don't know which approach would be correct to use here (for the purpose of, for example, putting error bars on a graph). Intuitively I'm drawn to the first method, because it seems to incorporate the most information about the data in the actual standard deviation calculation. I'm wary, though, that doing this could be be implicitly making some assumption about the structure of the data, such as homogeneity, which may not actually hold.

What approach is generally regarded as correct, and what assumptions about the structure of the data does it imply? Is there another, more correct method (or another method that makes fewer assumptions) which I failed to list?

$\endgroup$
  • 2
    $\begingroup$ The assertion that "the mean of two means is the same as the mean of all the points that went in" is simply false in the general case. I believe this only holds true when each "sub mean" includes an equal number of values. $\endgroup$ – Brian Mar 29 at 15:39
  • $\begingroup$ Thanks for the heads up, I'll edit the question $\endgroup$ – realityChemist Mar 29 at 15:44
2
$\begingroup$

If you consider your shrinkage estimates as samples from distributions with a common variance then the pooled estimate of the common variance is $$ s^2=\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2} $$

In this expression you have a sample of size $n_1$ with sample variance $s_1^2$ and a sample of size $n_2$ with sample variance $s_2^2$

If I understand your data, you have $n_1=2$ for widget 1 and $n_2=2$ for widget 2 giving $$ s^2=\frac{s_1^2+s_2^2}{2} $$ so actually the variance is the average of the individual variances, in this case. The standard deviation is the square root of the variance.

This link may be helpful.

$\endgroup$
  • $\begingroup$ I'm a bit confused by this (from the link): "You can only use the above formulas if the standard deviations for the two groups are the same (this is because it would otherwise be violating the assumption of homogeneity of variances." [sic] If the two standard deviations were the same, wouldn't these formulas simplify to tautologies, $s^2 = s^2$? Are they trying to say that the population standard deviations need to be the same in order to use this for sample standard deviations? $\endgroup$ – realityChemist Mar 29 at 17:06
  • 1
    $\begingroup$ @realityChemist This is what I meant when saying you have samples from a distribution with a common variance. All your samples are estimates of that common variance. I can't vouch for the exact content of the link I'm afraid, it was only added to provide a starting reference, terminology and a stepping stone to further searching around if need be. $\endgroup$ – PM. Mar 29 at 17:15
2
$\begingroup$

Before I answer your question: In general it is not true that the mean of two means is the mean of all points. consider the example $avg(avg(0,0,0),avg(1,1)) = 0.5 \neq 0.4 = avg(0,0,0,1,1)$.

Regarding the standard deviation: Only your first method actually makes sense, because the other methods do in general not coincide with the definition of the standard deviation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.