2
$\begingroup$

Suppose I use Welford's algorithm to compute the standard deviation of multiple sets of values. I only store all the n, mean and M2 results that the algorithm calculates, thus I have these three results per set.

Is it possible to calculate the standard deviation over all samples in all sets, by using only the stored results? Thus I'm looking for a way to combine the n, mean and M2 results. For n and mean this is trivial, but I don't know how to combine the M2 values.

Note: this would be easy when I would use the naive sum of squares algorithm, but wikipedia recommends against using this algorithm in practice.

Note2: I cannot store all values in all sets, I'm writing software that only gets one chance to do something with each value, thus the algorithm needs to be 'incremental' or 'online'.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.