This may end up being a question more about scientific best practice than anything else, but I think this is the right community to ask it in to get the insight I'm looking for.
Say I have two little square widgets made out of a material that shrinks when it gets wet. I want to know by how much. I measure the length of the widgets along two lines each (because they're not shaped perfectly and my measurement technique isn't perfect), before and after soaking them with water. I come back with data that looks like this:
Widget Measurement Before After Shrinkage 1 1 1.898 1.722 0.176 1 2 1.904 1.737 0.167 2 1 2.003 1.763 0.240 2 2 2.029 1.843 0.186
Now, I can calculate the overall mean without worrying too much in this case, since the mean of two means is the same as the mean of all the points that went in as long as each mean has the same number of samples, which in this case they do. So:
avg(0.176,0.167,0.240,0.186) = 0.192 = avg(avg(0.176,0.167),avg(0.240,0.186))
However, this type of relation is not true for the standard deviation. There are several approaches that immediately present themselves to me as options for finding an overall standard deviation for this dataset:
- Use all of the data at once:
sd(0.176,0.167,0.240,0.186) = 0.033
- Get a standard deviation for each widget, and average them:
avg(sd(0.176,0.167),sd(0.240,0.186)) = 0.022
- Get the average for each widget, and take the standard deviation of the two:
sd(avg(0.176,0.167),avg(0.240,0.186)) = 0.029
Now, maybe it's just confusion on my part as to the meaning of a standard deviation, but I don't know which approach would be correct to use here (for the purpose of, for example, putting error bars on a graph). Intuitively I'm drawn to the first method, because it seems to incorporate the most information about the data in the actual standard deviation calculation. I'm wary, though, that doing this could be be implicitly making some assumption about the structure of the data, such as homogeneity, which may not actually hold.
What approach is generally regarded as correct, and what assumptions about the structure of the data does it imply? Is there another, more correct method (or another method that makes fewer assumptions) which I failed to list?