# What am I reinventing? RE: Linear regression modeling for frequency of discrete events

I'm looking to model the frequency of events to quantify how much that frequency is increasing or decreasing. For the sake of concreteness think of the events as web page hits for several low traffic web sites, and I would like to compare how much they are "trending" up or down relative to one another. My main question is what am I reinventing?

In broad strokes this is the what I'm thinking. I have a set of events at a set of times $E = \{t_1,...t_N\}$ with $t_i < 0$. From these I have an event distribution function which is the sum of Dirac delta functions. $$\Phi(t) = K\sum_{i=1}^N\delta(t-t_i)$$ where K is some normalizing factor. I would like to model this like a linear regression with $L(t) = at + b$ by minimizing $\Vert{L-\Phi}\Vert$. Older events should be weighted less, so my inner product measure would be something like: $$d\omega = e^{kt}dt$$ Before I started digging out the details of this (the normalization, the inner product measure, etc.) I got the nagging suspicion that this has been done before :) So my question is -- where can I read about the established theory, practice, and terminology for this type of problem? Any suggestions about how to either rephrase the title of this question or reformulate the problem are appreciated as well.

• What do you exactly mean with the inner product measure? Do you mean that $\Vert{L-\Phi}\Vert^2=\int_{-\infty}^{0}(L(t)-\Phi(t))^2 e^{kt} dt$? Jan 29 '13 at 10:06
• @Rashkolnikov: yes.
– mjhm
Jan 29 '13 at 15:15

I think you are just looking for the weighted least squares method.

In the wiki article, a discrete version is tackled but I think it's rather easy to adapt this to the continuous case. What you want is to minimise

$$\int_{-\infty}^{0}\left(L(t)-\Phi(t)\right)^2 e^{kt}dt = \int_{-\infty}^{0}\left(at+b-\Phi(t)\right)^2 e^{kt}dt$$

As a first step, you can look for critical points by differentiating w.r.t. $a$ and $b$ and equating to zero. This gives the following set of equations

$$\begin{eqnarray} a \int_{-\infty}^{0}t^2 e^{kt}dt + b \int_{-\infty}^{0}t e^{kt}dt & = & \int_{-\infty}^{0}\Phi(t) t e^{kt}dt \\ a \int_{-\infty}^{0}t e^{kt}dt + b \int_{-\infty}^{0}e^{kt}dt & = & \int_{-\infty}^{0}\Phi(t) e^{kt}dt \end{eqnarray}$$

The integrals with $\Phi(t)=K\sum_{i=1}^N \delta(t-t_i)$ will reduce to sums while the other integrals can be worked out easily (they are just values of the gamma function or factorial).

$$\begin{eqnarray} -\frac{2}{k^3}a + \frac{1}{k^2}b & = & K\sum_{i=1}^N t_i e^{kt_i} \\ \frac{1}{k^2}a - \frac{2}{k} b & = & K\sum_{i=1}^N e^{kt_i} \end{eqnarray}$$

This linear system can be easily solved to give explicit formulas for $a$ and $b$.

EDIT: I just thought about something and I think it's not logical or natural to make a linear fit of $\Phi(t)$. One should try to fit the cumulative funtion i.e. $C(t)=\int_{-\infty}^t \Phi(u)du$ as this gives the count of events. But since I am not 100% sure what you want to achieve, I leave this as an additional comment. It's easy to adapt the above formulas for $C(t)$.

• Thanks for your helpful answer. You're statement that "it's not logical or natural to make a linear fit of $\phi(t)$" means that you recognize how this could be different from the usual least squares fitting. I'm ultimately interested in knowing whether the frequency of events is increasing or decreasing -- this should be indicated by whether the resulting "a" value is positive or negative. I'm also interested in comparing different sequences of events to see which are "trending" according to some common normalization.
– mjhm
Jan 29 '13 at 15:57
• The cumulative function seems like an interesting direction to pursue. To get "trend" meaning from that, I'm guessing that I would want to do a quadratic approximation, and examine it's second derivatives. (Though I still have the feeling that some smart mathematician has thought much more deeply about this than me.)
– mjhm
Jan 29 '13 at 16:04