# Let $(X_t)$ be a continuous-time Markov chain and $\tau$ the first jump time. Compute $\mathbb E_x [a^{\tau} \phi (X_\tau)]$

Let $$(X_t)$$ be a continuous-time Markov chain such that

• The state space $$V$$ is finite and endowed with discrete topology.

• The infintesimal generator is $$L: V^2 \to \mathbb R$$.

Let

• $$\alpha \in (0,1)$$.

• $$\phi$$ be a function from $$V$$ to $$\mathbb R_+$$.

• $$\tau$$ is the first jump time, i.e. the first time that the chain makes a transition to a new state.

I would like to ask how to compute $$\alpha = \mathbb E_x [a^{\tau} \phi (X_\tau)]$$ where $$\mathbb E_x := \mathbb E_x [ \cdot | X_0 = x ]$$.

My attempt:

It's well-known that given $$X_0$$, $$\tau$$ is exponentially distributed with parameter $$-L(X_0,X_0)$$. Then

$$\alpha = \mathbb E_x [a^{\tau} \phi (X_\tau)] = -\int_0^\infty a^s L(x,x)\phi (X_s) e^{-sL(x,x)} \mathrm{d}s$$

I'm stuck because there is $$s$$ inside $$\phi(X_s)$$. Could you please elaborate on how to compute $$\alpha$$?

Thank you so much!

• The issue in the calculation is using the formula$$E(g(τ))=\int g(t)\,\mathrm dF_τ(t),$$ which requires that $g$ be a deterministic measurable function of $τ$, but $Χ_τ$ is not deterministically determined by $τ$.
May 16 '20 at 13:12
• @Saad you meant that it's hard to find a closed form of $\alpha$? May 16 '20 at 13:20
• I'm not so familiar with continuous-time Markov chains, but I think there should exist a result similar to that in How $h(z)=\color{blue}{\alpha}\sum_yp_{z y}h(y)$ follows from Markov property?.
• Hi @Saad, I can not believe that I apply your suggestion successfully to solve for $\alpha$. Everything combines perfectly. Math is beautiful. Infinite thanks for you :)))))) May 16 '20 at 13:59
The issue in the calculation is using the formula$$E(g(τ))=\int g(t)\,\mathrm dF_τ(t),$$ which requires that $$g$$ be a deterministic measurable function of $$τ$$, but $$Χ_τ$$ is not deterministically determined by $$τ$$.
I'm not so familiar with continuous-time Markov chains, but I think there should exist a result similar to that in How $$h(z)=\color{blue}{\alpha}\sum_yp_{z y}h(y)$$ follows from Markov property?.