0
$\begingroup$

Say I have the following list of numbers, each corresponding to a single day:

list = [1,   6,    7,    12,   5]

I can calculate the daily, rolling standard deviations between each as:

stds = [NaN, 3.54, 0.71, 3.54, 4.95]

Is it now possible to calculate the standard deviation across all values of list, using only the values of stds?

$\endgroup$
5
  • $\begingroup$ Not sure I understand the values in stds. The standard deviation of the values $1,2$ is not $3.54$, whether you divide by $n$ or by $n-1$. What exactly is the $3.54$? Also...regardless of what your numbers mean, why isn't the last term in stds the desired value for the entire list? $\endgroup$
    – lulu
    Jan 16, 2019 at 13:09
  • $\begingroup$ @lulu Sorry, the second number was supposed to be 6, not 2. stds contains the standard deviations between each successive pair of numbers. The first value is blank, since there is no number before 1. The second value is std(1, 6), the third value is std(6, 7), etc. $\endgroup$
    – KOB
    Jan 16, 2019 at 13:17
  • $\begingroup$ Ok, but then the answer is "obviously not". The list $[1,6,7,12,19]$ has the same pairwise standard deviations but not the same total standard deviation. $\endgroup$
    – lulu
    Jan 16, 2019 at 13:22
  • $\begingroup$ Note that you can calculate standard deviation incrementally using just three stored values: the number of data observed, the sum of the data, and the sum of their squares. This seems to me a much more useful trick, since storing the list of "daily rolling standard deviations" takes about the same resources as storing the numbers themselves. $\endgroup$
    – David K
    Jan 17, 2019 at 6:11
  • $\begingroup$ @DavidK unfortunately the only data I receive is the pairwise standard deviations, and this cannot be changed as of now. $\endgroup$
    – KOB
    Jan 17, 2019 at 7:26

1 Answer 1

1
$\begingroup$

The rolling pairwise standard deviations don't give you enough information to get the total standard deviation. The two lists $$[0,1,0,1]\quad \&\quad [0,1,2,3]$$ both have the pairwise standard deviations $$\Big[\frac 1{\sqrt 2},\,\frac 1{\sqrt 2},\,\frac 1{\sqrt 2}\Big]$$

But the total standard deviation of the first is $.577$ and the standard deviation of the second is $1.29$.

Note: in all cases we are using the sample standard deviation (i.e. we divide by $n-1$ not $n$).

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .