# Linear regression of time series data - moving linear regression

Situation

A suitable analogy for my real-world problem would be a shop - customers arrive, spend a random amount of time in the shop and leave. The arrival behaviour of customers follows a Poisson distribution in that the inter-arrival times are random and independent of one another.

Whenever a customer arrives and whenever a customer leaves, the current throughput (customers leaving per second) and number of customers currently in the shop are recorded.

Problem

I want to be able to plot these pairs of values of throughput (Y axis) and the number of customers (X axis) on a virtual plane and find the line of best fit through the points to determine the gradient.

However, as this is time series data and over time more and more points will be plotted, I need some way of giving greater importance to more recent points, something like what is done in the rolling average.

Question

The idea I had was to sample departure events in fixed time windows to determine the throughput. After each sampling window, a value of the throughput and number of customers in the shop at the instant the window ends would be plotted on a virtual plane. Standard linear regression techniques (e.g. least squares) could then be deployed to determine the gradient of the line of best fit.

However, as this is time series data, there must be a way to limit the number of points plotted on the graph or lower the significance of points exponentially with time, like a rolling average. What is the best way to achieve this?

Secondly, some smoothing will no-doubt be required to stabilise the result. How could this be done?

• Why not use weights, that decrease (say, exponentially), the further you go into the past? That is, use a standard linear regression with weights $w_i=e^{-\lambda i}$ where $w_i$ is the weight of the $i$-th measurement, and the most recent measurement is $i=1$. $\lambda$ is a parameter that determines how much you want to bias your estimate towards more recent data. – yohBS Mar 30 '15 at 10:15
• That sounds perfect - thanks. Maybe it could even be extended with $\lambda_{i}$ which is derived from the number of events observed in each window such that windows with many events with have a greater influence than those with fewer events, thereby emphasising statistically significant results and smoothing over low-level noise. – SlappyTheFish Mar 30 '15 at 10:52
• The only problem I can see with this is that it would require all the points to be stored and recalculated every time. If there was a way to simply update the previous calculation of the gradient with a new pair of values then that would be perfect. – SlappyTheFish Mar 30 '15 at 10:55
• I don't know if this can be done in a simple way, but as a start I'd note that least-square fitting is done by calculating the inverse of a matrix of the form $A^TA$ where $A$ is a matrix constructed from the observations. Updating the inverse in each incremental step might be done via the Sherman-Morrison formula – yohBS Mar 30 '15 at 12:31