0
$\begingroup$

I am currently working through the book Introduction to Stochastic Processes with R by Dobrow, and am confused what the difference between the transition probabilities of an infinitesimal generator matrix, and the entries of the transition function. In the book, he denotes that the transition probabilities can be found using the generator matrix in the following way:

He defines $q_i$ as the holding time which is given as $$q_i = \displaystyle \sum_{j\neq i}q_{ij}$$ where $i$ and $j$ are states in a continuous time finite state Markov chain, and $q_{ij}$ is the transition rate from state $i$ to $j$.

Using this fact, he states that the transition probability $p_{ij} = \frac{q_{ij}}{q_{i}}$, and that the matrix consisting of these probabilities is called the transition matrix for the embedded chain denoted as $\tilde{\mathbf{P}}$

He also states that the transition function of a generator matrix $\mathbf{Q}$ can be found by finding the matrix exponential of $\mathbf{Q}$, given as $$ \mathbf{P}(t) = e^{t\mathbf{Q}} $$

What is the difference between $\tilde{\mathbf{P}}$ and $𝐏(𝑡)$ for a continuous time finite state Markov chain, and when should the embedded chain matrix be used over the transition function matrix?

$\endgroup$
1
$\begingroup$

The transition rates $q_{ij}$ determine the time evolution of the process. The embedded chain only records the order in which states are visited. For example, the generator matrix $Q$ satisfies $[e^{tQ}]_{ij} = \mathbb P(X(t)=j\mid X(0)=i)$. You can think of the embedded chain as a semi-Markov process with holding times identically equal to one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.