Say I have a continuous time markov chain, time homogeneous $X$ with a few states (say, 2). I want to know the distribution of where $X$ is at time $t$, call it $\mu_t$, which will be a vector of 2 elements since there are two states. From the book Grimmet and Stirzaker, we have the Chapman Kolmogorov equations:

$$\mu_{t+s}=\mu_t*P_s, \forall s,t \ge 0$$

where $P_s$ is the infinitesimal probability transition matrix at time $s$.

I know we also have the Kolmogorov forward and Kolmogorov backward equations which tell us how $P_t$ evolves with time.

I'd like to when it is necessary to use the KFE equations. In every Markov Chain problem, we are given either:

  • The probability transition matrix, $P_t$ explicitly, OR
  • The generator matrix $G$

IF I have the generator matrix $G$ then like it shows on page 258, I can easily get the $P_t$ matrix by $p_ii=1+g_{ii}*h+o(h)$ and $p_ij=g_{ij}*h+o(h)$ (as shown on page 258)

and if I have $P_t$ matrix, then I can always use the Chapman-Kolmogorov equation in order to get $\mu_t$ at an instant in time $t$: $\mu_{t}=\mu_0*P_t$ (as I showed above). Is that correct?

Then, given $\mu_0$, the initial state distribution, I don’t see any reason to solve the KFE equations. When would I need to solve them?

  • 1
    $\begingroup$ You have tagged this as stochastic-pde. Is there a stochastic-pde somewhere in your question? $\endgroup$ – user608030 Dec 30 '18 at 1:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.