2
$\begingroup$

Let $(X_t)$ be a continuous-time Markov chain such that

  • The state space $V$ is finite and endowed with discrete topology.

  • The infintesimal generator is $L: V^2 \to \mathbb R$.

Let

  • $\alpha \in (0,1)$.

  • $\phi$ be a function from $V$ to $\mathbb R_+$.

  • $\tau$ is the first jump time, i.e. the first time that the chain makes a transition to a new state.

I would like to ask how to compute $$\alpha = \mathbb E_x [a^{\tau} \phi (X_\tau)]$$ where $\mathbb E_x := \mathbb E_x [ \cdot | X_0 = x ]$.


My attempt:

It's well-known that given $X_0$, $\tau$ is exponentially distributed with parameter $-L(X_0,X_0)$. Then

$$\alpha = \mathbb E_x [a^{\tau} \phi (X_\tau)] = -\int_0^\infty a^s L(x,x)\phi (X_s) e^{-sL(x,x)} \mathrm{d}s$$

I'm stuck because there is $s$ inside $\phi(X_s)$. Could you please elaborate on how to compute $\alpha$?

Thank you so much!

$\endgroup$
6
  • $\begingroup$ The issue in the calculation is using the formula$$E(g(τ))=\int g(t)\,\mathrm dF_τ(t),$$ which requires that $g$ be a deterministic measurable function of $τ$, but $Χ_τ$ is not deterministically determined by $τ$. $\endgroup$
    – Saad
    May 16 '20 at 13:12
  • $\begingroup$ @Saad you meant that it's hard to find a closed form of $\alpha$? $\endgroup$
    – Akira
    May 16 '20 at 13:20
  • $\begingroup$ I'm not so familiar with continuous-time Markov chains, but I think there should exist a result similar to that in How $h(z)=\color{blue}{\alpha}\sum_yp_{z y}h(y)$ follows from Markov property?. $\endgroup$
    – Saad
    May 16 '20 at 13:32
  • $\begingroup$ Thank you so much @Saad! You got me an idea of apply that formula to this case :)) $\endgroup$
    – Akira
    May 16 '20 at 13:34
  • $\begingroup$ Hi @Saad, I can not believe that I apply your suggestion successfully to solve for $\alpha$. Everything combines perfectly. Math is beautiful. Infinite thanks for you :)))))) $\endgroup$
    – Akira
    May 16 '20 at 13:59
0
$\begingroup$

Thank you so much for @Saad's invaluable comment! I post it here to close this question:


The issue in the calculation is using the formula$$E(g(τ))=\int g(t)\,\mathrm dF_τ(t),$$ which requires that $g$ be a deterministic measurable function of $τ$, but $Χ_τ$ is not deterministically determined by $τ$.

I'm not so familiar with continuous-time Markov chains, but I think there should exist a result similar to that in How $h(z)=\color{blue}{\alpha}\sum_yp_{z y}h(y)$ follows from Markov property?.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.