For an inhomogeneous Poisson process with instantaneous rate $\lambda(t)$, the log likelihood of observing events at times $t_1,\ldots,t_n$ in the time interval $[0,T)$ is given by

$ \sum_i \mathrm{log}\lambda(t_i) - \int_0^T \lambda(t) dt$

I am told this can be derived by taking the limit of the discrete-time case as the bin width $\Delta t$ goes to $0$:

$ \sum_i \mathrm{log}(\lambda(t_i)\Delta t) + \sum_{t\notin \{t_1,\ldots, t_n\}} \mathrm{log}(1-\lambda(t) \Delta t)$

For the second term, it is clear how this limit works if we take the Taylor expansion:

$\mathrm{log}(1-\epsilon) \approx -\epsilon$

However for the first term, it is not clear to me how one goes from $\sum_i \mathrm{log}\lambda(t_i) + \mathrm{log} \Delta t$ to $\sum_i\mathrm{log}\lambda(t_i)$ as $\Delta t \rightarrow 0$. Shouldn't the $\mathrm{log}\Delta t$ terms go to $-\infty$?


I think I understand now where my confusion was coming from. If we look at the probability, instead of log probability, then as $\Delta t$ becomes small the probability approaches:

$ \Delta t^n \prod_{i=1}^n \lambda(t_i) \mathrm{exp}\left(-\int_0^T\lambda(t)dt\right)$

The probability of an event happening in the interval:

$[t_1 - \Delta t/2, t_1 + \Delta t/2] \times [t_2 - \Delta t/2, t_2 + \Delta t/2] \times \ldots \times [t_n - \Delta t/2, t_n + \Delta t/2]$

should scale as $\Delta t^n$ for small $\Delta t$, so while the probability of events happening at exactly those times goes to zero (as it must), the density does not.

  • $\begingroup$ Hi David, do you have a citation for the likelihood? I arrived at the same likelihood by reason directly from the entropy of a Poisson process given by McFadden. $\endgroup$ – Neil G Apr 15 '13 at 19:46

Disclaimer: This is, so far, one of my most downvoted answers on the site. Needless to say, it is perfectly correct, and it answers the question as formulated at the time. The downvotes might be due to extra-mathematical reasons. Happy reading!

The density with respect to the Lebesgue measure $\mathrm dt_1\mathrm dt_2\cdots\mathrm dt_n$ of the distribution of the $n$ first events of the Poisson process is $$ \lambda(t_1)\mathrm e^{-\Lambda(t_1)}\cdot\lambda(t_2)\mathrm e^{-(\Lambda(t_2)-\Lambda(t_1))}\cdots\lambda(t_n)\mathrm e^{-(\Lambda(t_n)-\Lambda(t_{n-1}))}=\lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(t_n)}, $$ on the set $0\lt t_1\lt t_2\lt\cdots\lt t_n$, where, for every $t\geqslant0$, $$ \Lambda(t)=\int_0^t\lambda(s)\mathrm ds. $$ Let $T\gt0$ and let $A_n^T$ denote the event that exactly $n$ events of the Poisson process occur in $(0,T]$. Then $A_n^T$ happens if and only if the $n$ first events of the Poisson process happen at some times $0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T$ and if there is no further event in $(t_n,T]$. Thus, the density of the distribution of the $n$ first events of the Poisson process restricted to the event $A_n^T$ is $$ \lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(t_n)}\cdot\mathrm e^{-(\Lambda(T)-\Lambda(t_n))} $$ on the set $0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T$, that is, $$ \lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(T)}\cdot\mathbf 1_{0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T}. $$ To sum up, the quantity in the question is the log-likelihood of the $n$ first events of the Poisson process, restricted to $A_n^T$.

One can recover this result by discretization, but not in the limit you suggest. Rather, for every $s\gt0$, consider an independent Bernoulli process $X^s=(X^s_i)_{i\geqslant1}$ such that $p_i^s=\mathbb P(X_i^s=1)$ is $p_i^s=\Lambda(is)-\Lambda((i-1)s)$. Call $(T^s_k)_{k\geqslant1}$ the times when the Bernoulli process $X^s$ is $1$, defined recursively by $T^s_0=0$ and, for every $k\geqslant0$, $$ T^s_{k+1}=\inf\{i\geqslant T^s_k+1\mid X^s_i=1\}. $$ For every $N\geqslant n$, call $A_n^{s,N}$ the event that exactly $n$ Bernoulli random variables in the process $X^s$ up to time $N$ are $1$, thus $A_n^{s,N}=[T^s_n\leqslant N\lt T^s_{n+1}]$. Then the density of $(T^s_k)_{1\leqslant k\leqslant n}$ restricted to the event $A_n^{s,N}$ is $$ \mathbb P((T^s_k)_{1\leqslant k\leqslant n}=(i_k)_{1\leqslant k\leqslant n},A_n^{s,N})=\prod_{k=1}^np^s_{i_k}\cdot\prod_*(1-p^s_i), $$ where $*$ denotes the product over every $1\leqslant i\leqslant N$ except the times $i_k$ for $1\leqslant k\leqslant n$. Call the RHS $R_n^{s,N}(\mathbf i)$ where $\mathbf i=(i_k)_{1\leqslant k\leqslant n}$, then $$ R_n^{s,N}(\mathbf i)=\prod_{k=1}^n\frac{p^s_{i_k}}{1-p^s_{i_k}}\cdot\prod_{i=1}^N(1-p^s_i). $$ Now consider the limit $$ s\to0,\qquad sN\to T\in(0,\infty),\qquad si_k\to t_k,\qquad n\ \text{fixed}. $$ Then, $p^s_{i_k}=\lambda(t_k)s+o(s)$ and $p^s_i=\lambda(is)s+o(s)$ hence $$ \frac{p^s_{i_k}}{1-p^s_{i_k}}\sim\lambda(t_k)s,\quad\prod_{i=1}^N(1-p^s_i)\sim\exp\left(-s\sum_{i=1}^N\lambda(si)\right)\sim\mathrm e^{-\Lambda(T)}. $$ Finally, in the limit considered, $$ R_n^{s,N}(\mathbf i)\sim R_n^T(\mathbf t)\cdot s^n, $$ where $\mathbf t=(t_k)_{1\leqslant k\leqslant n}$, and $$ R_n^T(\mathbf t)=\prod_{k=1}^n\lambda(t_k)\cdot\mathrm e^{-\Lambda(T)}. $$ The log-likelihood in the question is the logarithm of $R_n^T(\mathbf t)$.

  • 6
    $\begingroup$ Why was this downvoted so much? It looks fine to me. $\endgroup$ – Ben Lansdell May 25 '15 at 18:35
  • $\begingroup$ Would someone please provide a citation for this or other proofs of this result? I recently read a different proof in Thompson's Point Process Models with Applications to Safety and Reliability, but would like to see the intuition provided by different authors. $\endgroup$ – Tom Hallward Feb 2 '19 at 0:24
  • $\begingroup$ @TomHallward Any textbook on Poisson processes should explain this, or variants of it. Say, what do you find in all this, which would not be a direct consequence of the definitions? $\endgroup$ – Did Feb 2 '19 at 9:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.