# Log likelihood of a realization of a Poisson process?

For an inhomogeneous Poisson process with instantaneous rate $\lambda(t)$, the log likelihood of observing events at times $t_1,\ldots,t_n$ in the time interval $[0,T)$ is given by

$\sum_i \mathrm{log}\lambda(t_i) - \int_0^T \lambda(t) dt$

I am told this can be derived by taking the limit of the discrete-time case as the bin width $\Delta t$ goes to $0$:

$\sum_i \mathrm{log}(\lambda(t_i)\Delta t) + \sum_{t\notin \{t_1,\ldots, t_n\}} \mathrm{log}(1-\lambda(t) \Delta t)$

For the second term, it is clear how this limit works if we take the Taylor expansion:

$\mathrm{log}(1-\epsilon) \approx -\epsilon$

However for the first term, it is not clear to me how one goes from $\sum_i \mathrm{log}\lambda(t_i) + \mathrm{log} \Delta t$ to $\sum_i\mathrm{log}\lambda(t_i)$ as $\Delta t \rightarrow 0$. Shouldn't the $\mathrm{log}\Delta t$ terms go to $-\infty$?

Edit

I think I understand now where my confusion was coming from. If we look at the probability, instead of log probability, then as $\Delta t$ becomes small the probability approaches:

$\Delta t^n \prod_{i=1}^n \lambda(t_i) \mathrm{exp}\left(-\int_0^T\lambda(t)dt\right)$

The probability of an event happening in the interval:

$[t_1 - \Delta t/2, t_1 + \Delta t/2] \times [t_2 - \Delta t/2, t_2 + \Delta t/2] \times \ldots \times [t_n - \Delta t/2, t_n + \Delta t/2]$

should scale as $\Delta t^n$ for small $\Delta t$, so while the probability of events happening at exactly those times goes to zero (as it must), the density does not.

-
Hi David, do you have a citation for the likelihood? I arrived at the same likelihood by reason directly from the entropy of a Poisson process given by McFadden. – Neil G Apr 15 '13 at 19:46

The density with respect to the Lebesgue measure $\mathrm dt_1\mathrm dt_2\cdots\mathrm dt_n$ of the distribution of the $n$ first events of the Poisson process is $$\lambda(t_1)\mathrm e^{-\Lambda(t_1)}\cdot\lambda(t_2)\mathrm e^{-(\Lambda(t_2)-\Lambda(t_1))}\cdots\lambda(t_n)\mathrm e^{-(\Lambda(t_n)-\Lambda(t_{n-1}))}=\lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(t_n)},$$ on the set $0\lt t_1\lt t_2\lt\cdots\lt t_n$, where, for every $t\geqslant0$, $$\Lambda(t)=\int_0^t\lambda(s)\mathrm ds.$$ Let $T\gt0$ and let $A_n^T$ denote the event that exactly $n$ events of the Poisson process occur in $(0,T]$. Then $A_n^T$ happens if and only if the $n$ first events of the Poisson process happen at some times $0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T$ and if there is no further event in $(t_n,T]$. Thus, the density of the distribution of the $n$ first events of the Poisson process restricted to the event $A_n^T$ is $$\lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(t_n)}\cdot\mathrm e^{-(\Lambda(T)-\Lambda(t_n))}$$ on the set $0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T$, that is, $$\lambda(t_1)\lambda(t_2)\cdots\lambda(t_n)\cdot\mathrm e^{-\Lambda(T)}\cdot\mathbf 1_{0\lt t_1\lt t_2\lt\cdots\lt t_n\lt T}.$$ To sum up, the quantity in the question is the log-likelihood of the $n$ first events of the Poisson process, restricted to $A_n^T$.
One can recover this result by discretization, but not in the limit you suggest. Rather, for every $s\gt0$, consider an independent Bernoulli process $X^s=(X^s_i)_{i\geqslant1}$ such that $p_i^s=\mathbb P(X_i^s=1)$ is $p_i^s=\Lambda(is)-\Lambda((i-1)s)$. Call $(T^s_k)_{k\geqslant1}$ the times when the Bernoulli process $X^s$ is $1$, defined recursively by $T^s_0=0$ and, for every $k\geqslant0$, $$T^s_{k+1}=\inf\{i\geqslant T^s_k+1\mid X^s_i=1\}.$$ For every $N\geqslant n$, call $A_n^{s,N}$ the event that exactly $n$ Bernoulli random variables in the process $X^s$ up to time $N$ are $1$, thus $A_n^{s,N}=[T^s_n\leqslant N\lt T^s_{n+1}]$. Then the density of $(T^s_k)_{1\leqslant k\leqslant n}$ restricted to the event $A_n^{s,N}$ is $$\mathbb P((T^s_k)_{1\leqslant k\leqslant n}=(i_k)_{1\leqslant k\leqslant n},A_n^{s,N})=\prod_{k=1}^np^s_{i_k}\cdot\prod_*(1-p^s_i),$$ where $*$ denotes the product over every $1\leqslant i\leqslant N$ except the times $i_k$ for $1\leqslant k\leqslant n$. Call the RHS $R_n^{s,N}(\mathbf i)$ where $\mathbf i=(i_k)_{1\leqslant k\leqslant n}$, then $$R_n^{s,N}(\mathbf i)=\prod_{k=1}^n\frac{p^s_{i_k}}{1-p^s_{i_k}}\cdot\prod_{i=1}^N(1-p^s_i).$$ Now consider the limit $$s\to0,\qquad sN\to T\in(0,\infty),\qquad si_k\to t_k,\qquad n\ \text{fixed}.$$ Then, $p^s_{i_k}=\lambda(t_k)s+o(s)$ and $p^s_i=\lambda(is)s+o(s)$ hence $$\frac{p^s_{i_k}}{1-p^s_{i_k}}\sim\lambda(t_k)s,\quad\prod_{i=1}^N(1-p^s_i)\sim\exp\left(-s\sum_{i=1}^N\lambda(si)\right)\sim\mathrm e^{-\Lambda(T)}.$$ Finally, in the limit considered, $$R_n^{s,N}(\mathbf i)\sim R_n^T(\mathbf t)\cdot s^n,$$ where $\mathbf t=(t_k)_{1\leqslant k\leqslant n}$, and $$R_n^T(\mathbf t)=\prod_{k=1}^n\lambda(t_k)\cdot\mathrm e^{-\Lambda(T)}.$$ The log-likelihood in the question is the logarithm of $R_n^T(\mathbf t)$.