# Log likelihood of a realization of a Poisson process?

For an inhomogeneous Poisson process with instantaneous rate $\lambda(t)$, the log likelihood of observing events at times $t_1,\ldots,t_n$ in the time interval $[0,T)$ is given by

$\sum_i \mathrm{log}\lambda(t_i) - \int_0^T \lambda(t) dt$

I am told this can be derived by taking the limit of the discrete-time case as the bin width $\Delta t$ goes to $0$:

$\sum_i \mathrm{log}(\lambda(t_i)\Delta t) + \sum_{t\notin \{t_1,\ldots, t_n\}} \mathrm{log}(1-\lambda(t) \Delta t)$

For the second term, it is clear how this limit works if we take the Taylor expansion:

$\mathrm{log}(1-\epsilon) \approx -\epsilon$

However for the first term, it is not clear to me how one goes from $\sum_i \mathrm{log}\lambda(t_i) + \mathrm{log} \Delta t$ to $\sum_i\mathrm{log}\lambda(t_i)$ as $\Delta t \rightarrow 0$. Shouldn't the $\mathrm{log}\Delta t$ terms go to $-\infty$?

Edit

I think I understand now where my confusion was coming from. If we look at the probability, instead of log probability, then as $\Delta t$ becomes small the probability approaches:

$\Delta t^n \prod_{i=1}^n \lambda(t_i) \mathrm{exp}\left(-\int_0^T\lambda(t)dt\right)$

The probability of an event happening in the interval:

$[t_1 - \Delta t/2, t_1 + \Delta t/2] \times [t_2 - \Delta t/2, t_2 + \Delta t/2] \times \ldots \times [t_n - \Delta t/2, t_n + \Delta t/2]$

should scale as $\Delta t^n$ for small $\Delta t$, so while the probability of events happening at exactly those times goes to zero (as it must), the density does not.

-
Hi David, do you have a citation for the likelihood? I arrived at the same likelihood by reason directly from the entropy of a Poisson process given by McFadden. –  Neil G Apr 15 '13 at 19:46

Disclaimer: This is, so far, one of my most downvoted answers on the site. Needless to say, it is perfectly correct, and it answers the question as formulated at the time. The downvotes might be due to extra-mathematical reasons. Happy reading!

The density with respect to the Lebesgue measure $\mathrm dt_1\mathrm dt_2\cdots\mathrm dt_n$ of the distribution of the $n$ first events of the Poisson process is $$\lambda(t_1)\mathrm e^{-\Lambda(t_1)}\cdot\lambda(t_2)\mathrm e^{-(\Lambda(t_2)-\Lambda(t_1))}\cdots\lambda(t_n)\mathrm e^{-(\Lambda(t_n)-\Lambda(t_{n-1}))}=\lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(t_n)},$$ on the set $0\lt t_1\lt t_2\lt\cdots\lt t_n$, where, for every $t\geqslant0$, $$\Lambda(t)=\int_0^t\lambda(s)\mathrm ds.$$ Let $T\gt0$ and let $A_n^T$ denote the event that exactly $n$ events of the Poisson process occur in $(0,T]$. Then $A_n^T$ happens if and only if the $n$ first events of the Poisson process happen at some times $0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T$ and if there is no further event in $(t_n,T]$. Thus, the density of the distribution of the $n$ first events of the Poisson process restricted to the event $A_n^T$ is $$\lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(t_n)}\cdot\mathrm e^{-(\Lambda(T)-\Lambda(t_n))}$$ on the set $0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T$, that is, $$\lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(T)}\cdot\mathbf 1_{0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T}.$$ To sum up, the quantity in the question is the log-likelihood of the $n$ first events of the Poisson process, restricted to $A_n^T$.

One can recover this result by discretization, but not in the limit you suggest. Rather, for every $s\gt0$, consider an independent Bernoulli process $X^s=(X^s_i)_{i\geqslant1}$ such that $p_i^s=\mathbb P(X_i^s=1)$ is $p_i^s=\Lambda(is)-\Lambda((i-1)s)$. Call $(T^s_k)_{k\geqslant1}$ the times when the Bernoulli process $X^s$ is $1$, defined recursively by $T^s_0=0$ and, for every $k\geqslant0$, $$T^s_{k+1}=\inf\{i\geqslant T^s_k+1\mid X^s_i=1\}.$$ For every $N\geqslant n$, call $A_n^{s,N}$ the event that exactly $n$ Bernoulli random variables in the process $X^s$ up to time $N$ are $1$, thus $A_n^{s,N}=[T^s_n\leqslant N\lt T^s_{n+1}]$. Then the density of $(T^s_k)_{1\leqslant k\leqslant n}$ restricted to the event $A_n^{s,N}$ is $$\mathbb P((T^s_k)_{1\leqslant k\leqslant n}=(i_k)_{1\leqslant k\leqslant n},A_n^{s,N})=\prod_{k=1}^np^s_{i_k}\cdot\prod_*(1-p^s_i),$$ where $*$ denotes the product over every $1\leqslant i\leqslant N$ except the times $i_k$ for $1\leqslant k\leqslant n$. Call the RHS $R_n^{s,N}(\mathbf i)$ where $\mathbf i=(i_k)_{1\leqslant k\leqslant n}$, then $$R_n^{s,N}(\mathbf i)=\prod_{k=1}^n\frac{p^s_{i_k}}{1-p^s_{i_k}}\cdot\prod_{i=1}^N(1-p^s_i).$$ Now consider the limit $$s\to0,\qquad sN\to T\in(0,\infty),\qquad si_k\to t_k,\qquad n\ \text{fixed}.$$ Then, $p^s_{i_k}=\lambda(t_k)s+o(s)$ and $p^s_i=\lambda(is)s+o(s)$ hence $$\frac{p^s_{i_k}}{1-p^s_{i_k}}\sim\lambda(t_k)s,\quad\prod_{i=1}^N(1-p^s_i)\sim\exp\left(-s\sum_{i=1}^N\lambda(si)\right)\sim\mathrm e^{-\Lambda(T)}.$$ Finally, in the limit considered, $$R_n^{s,N}(\mathbf i)\sim R_n^T(\mathbf t)\cdot s^n,$$ where $\mathbf t=(t_k)_{1\leqslant k\leqslant n}$, and $$R_n^T(\mathbf t)=\prod_{k=1}^n\lambda(t_k)\cdot\mathrm e^{-\Lambda(T)}.$$ The log-likelihood in the question is the logarithm of $R_n^T(\mathbf t)$.

-
Tried to undo it. But looking over it more closely, you didn't actually answer my question, you just restated it in a fancier way. I had a basic misunderstanding about stochastic processes that you didn't address. –  David Pfau Mar 29 '13 at 3:20
Punch in the face? Calm down. It's obviously a drop in the bucket of your otherwise sterling reputation. Take a few deep breaths and move on. –  David Pfau Apr 10 '13 at 15:26
I upvoted before reading it fully. After going through it I realized that it did not answer my question, but only restated it with different notation (and as a probability instead of a log probability) so I undid my upvote. I feel that no vote would be a more appropriate response, but at this point my vote is locked in, and I figured someone with a reputation as high as yours would not be bothered by a single downvote (obviously I was wrong). If your answer did inspire my edit, it was only by confirming that there was no mistake in my derivation and that my confusion was coming from elsewhere. –  David Pfau Apr 10 '13 at 16:38
Unhelpful. Expectedly. –  David Pfau Apr 10 '13 at 17:07
@Did Is your attitude the reason for the strong downvotes here(I am not implying anything), I just don't see how such a detailed answer could possibly inspire so many downvotes. –  this is much healthier Aug 25 '14 at 10:48