1
$\begingroup$

Take for example this dataset:

[1,1,1,1,1,1,1,1,1,200,1,1,1,1,....,1]

I want to calculate a running average on this with a certain window size for avg, let's say 10 most recent data point.

A regular average with a window of 10 (or whatever) would look something like this:

regular avg with window

The bump is when the 200 enters the calculation and the cliff is when it leaves the window.

I want it to look like this:

what I want it to look like

I believe this means I need to give more weight to recent data to reduce the impact of the 200 point bump over time.

What is a good way to do this? I'm looking for pointers in the right direction.

Thanks!

$\endgroup$
  • $\begingroup$ So apparently your idea is to apply a weight factor to recent values (for example, only the last value in the window). You could try it with values like 2 or 20 or something. Did you try it? $\endgroup$ – Matti P. Dec 11 '18 at 13:38
  • $\begingroup$ Let's consider the case where the value of $200$ is just about to leave the window. Then we have the numbers $200, 1, 1, \ldots ,1$ in the window (nine 1's). If we set a weight of $10$ for the latest value (the last 1 in the window), the window average is $$ \frac{200 + 9\times 1 + 10 \times 1}{9+10} \approx 11.5 $$ This is about half of the average that we would get without the weights. $\endgroup$ – Matti P. Dec 11 '18 at 13:45
  • $\begingroup$ Thanks @MattiP. I was thinking weighing all the previous values based on their age so that their influence would decay with age $\endgroup$ – Ben Dec 11 '18 at 14:04
1
$\begingroup$

What you might want to use as your weight given your graph is using a three-part piecewise function: $$ z=\begin{cases} 0 & x<0\\ 1-0.08x & 0\leq x\leq10 \\ 0 & 10\leq x \end{cases} $$ where $x$ denotes days past and $z$ denotes the weight. The first part prevents returning a value to weigh anything that happens, the second makes the linear regression and the third creates the cutoff at the end of the graph. After finding the weight, you calculate the value of a certain point as you would for any arithmetic mean. For example, for $2$ days past the $200$ value, you would find $z$ for the $200$ point as $1-0.08\cdot2=0.84$, and the weights for the $1$s being $1,0.92,0.76,0.68$ e.t.c.

$\endgroup$
1
$\begingroup$

Input sequence: $a_1, a_2, a_3, \dots$

Output $b_n$

$b_1 = a_1$

$b_{n+1} = .95 * b_n + .05 * a_{n+1}$

Here you give past readings a 95% weight and the current reading a 5% weight.

More generally,

$b_{n+1} = \alpha * a_{n+1} + (1-\alpha) * b_n \; \text{ with } 0 \lt \alpha \lt 1$

Your data looks a bit strange, so you might have to reset the filter when the big readings come through, so $b_n = 200$ and then start the average smoothing.

See wikipedia link on exponential smoothing.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.