# Incremental averageing

Is there a way to incrementally calculate (or estimate) the average of a vector (a set of numbers) without knowing their count in advance?

For example you have a = [4 6 3 9 4 12 4 18] and you want to get an estimate of the average but you don't have all the values at hand so you want to have a running average without keeping all the previous values available.

-
See the answers to this question on the sister site dsp.SE – Dilip Sarwate Feb 7 '12 at 15:46
Possible duplicate of this question – Chris Taylor Feb 7 '12 at 17:00

You need to keep at least two pieces of information: the number of terms so far and the running mean (or something equivalent to it).

Let's suppose the $n$th component of the vector is $a_n$ and the running mean up to this is $m_n$ so $$m_n= \frac{1}{n}\sum_{i=1}^n a_i.$$

Starting with $m_0=0$, you can use the obvious single pass $$m_n = \frac{(n-1)m_{n-1}+a_n}{n}$$ but precision errors are likely to be smaller if you use the equivalent $$m_n = m_{n-1} + \frac{a_{n}-m_{n-1}}{n}.$$

-
Can you elaborate on why precision errors are smaller in the second expression? Does the first expression contain a subtraction of nearly equal values? – Paul Nov 18 '13 at 16:02
If $n$ is large, it is likely that $(n-1)m_{n-1}$ will be much larger than $a_n$, so making the addition less precise – Henry Nov 18 '13 at 19:58
What if n is so big that (using finite precision float point numbers) $$\frac{a_n - m_{n-1}}{n}$$ gives us zero? – castarco Jan 8 '14 at 18:57
@castarco: Then given the available precision, the change in the mean is less that the smallest positive number. – Henry Jan 8 '14 at 19:18
@Henry yes, but we still have a problem. We reach that $n$, and one sample wont change the mean.. but what if we have $k$ new samples? I think the solution is to calculate the mean using a "tree" of averages of samples subsets. – castarco Jan 9 '14 at 16:35

yep easy to do - I call it the travelling mean :)

Note you will need to keep track of the 1. 'count' of values, 2. previous 'mean' and 3. the 'new value'. Algorithm is:

in words : ('previous mean' * '(count -1)') + 'new value') / 'count'

benefit you have a running mean can do the same with 'standard deviation' just a little more complex

-

Here is the general solution to the problem. We start calculating average from (m+1)-th element up to (n+1)-th element of the incoming data.
Giving that:

• $a_{n}$ is the n-th element of incoming data,
• $a_{m}$ is the element from which we start averaging,
• $\widehat{a_{n-m}}$ is the avarage from m-th to n-th element,
• $\widehat{a_{n+1-(m+1)}}$ is the avarage from (m+1)-th to (n+1)-th element

So, if we initially have $a_{m}, a_{m+1}, a_{m+2}, \ldots, a_{n}$, an average of $n-m$ elements can be easily calculated. That is $$\widehat{a_{n-m}} = \frac{a_{m}+a_{m+1}+a_{m+2}+\ldots+a_{n}}{n-m}$$

Next, when the $n+1$ elementh comes, we want to obtain average calculated of the same number of elements that is from $a_{m+1}$ to $a_{n+1}$:
$$\widehat{a_{(n+1)-(m+1)}} = \frac{a_{m+1} + a_{m+2} + \ldots + a_{n} + a_{n+1}}{(n+1) - (m+1)}$$

Then from first equation $$\widehat{a_{n-m}}(n-m) - a_{m} = a_{m+1}+a_{m+2}+\ldots+a_{n}$$

Substituting this to equation 2 $$\widehat{a_{(n+1)-(m+1)}} = \frac{\widehat{a_{n-m}}(n-m) - a_{m} + a_{n+1}}{n-m}$$

In order to dynamicaly calculate average for new element we need previous average $\widehat{a_{n-m}}$, first element of the previous average $a_{m}$, number of the elements we include in the average $n-m$.
I hope you will find this solution inspiring. Please write if you find any errors in the above reasoning.

-
I have to mention, that I didn't manage to find any practical appliaction for this avarage. I thought about using it for calculating complexity-efficient avarage with big ammount of data (let sat n-m = 1000), but in the end I gave up. If you know how to store $n-m$th element (for $a_{m}$) without using $n-m$th element array, please let me know. – Michal Wolodzko Feb 10 '14 at 16:18
So, what happens in the case you are going one step at a time and don't want to store any past information, like you have the previous average, now you have one new element? what else you need to keep, total count of elements and the first one? – Ali Feb 10 '14 at 16:20
Yes. You need at least to know how many elements you include in the average, the first element in the average and the average from the previous step. Note please, that $n-m$, that is the total count of elements in the average can be a constant number. So at least one piece of information you have given "for free" - you don't have to calculate it when new data arrives. – Michal Wolodzko Feb 12 '14 at 16:09