# Markov chain, Q matrix, jump matrix and invariant distribution

For the following Q matrix i want to find the jump matrix and the invariant distribution. $Q= \begin{pmatrix} -2 &1 &1 &0\\ 2 & -4 &1 &1\\ 1 &0 &-1 &0\\ 1 &1 &0 &-2 \end{pmatrix}$ I can find the jump matrix but am not sure how to find invariant distribution. Is it the same as discrete case?

-

Hint: Solve $\mu^TQ=0$ (instead of $m^TP=m$ in the discrete time case).

-
Thanks i'll try this as well to see if it comes out the same – Johnny Byr Oct 18 '12 at 23:51
You know what? It does. – Did Oct 19 '12 at 20:10

$\def\Id{\operatorname{Id}}$Computing the invariant distribution can be down in three steps:

1. Compute (as you allready did), the jump matrix $S$ (in the following $D$ will denote $Q$s diagonal part). $S = \Id - D^{-1}Q = \Id - \begin{pmatrix} 1 &-\frac 12 &-\frac 12 &0\\ -\frac 12 & 1 &-\frac 14 &-\frac 14\\ -1 &0 &1 &0\\ -\frac 12 &-\frac 12&0 &1 \end{pmatrix}= \begin{pmatrix} 0 &\frac 12 &\frac 12 &0\\ \frac 12 & 0 &\frac 14 &\frac 14\\ 1 &0 &0 &0\\ \frac 12 &\frac 12&0 &0 \end{pmatrix}$

2. Compute a right eigenvector $\varphi$ of $S$ for the eigenvalue 1, that is $\varphi$ fulfills $\varphi = \varphi S$, we will find $\varphi^t$, $S^t -\Id$ equals \begin{align*} \begin{pmatrix} -1 &\frac 12 & 1 & \frac 12\\ \frac 12 & -1 &0 &\frac 12\\ \frac 12&\frac 14 &-1 &0\\ 0 &\frac 14&0 &-1 \end{pmatrix} &\leadsto \begin{pmatrix} -1 &\frac 12 & 1 & \frac 12\\ 1 & -2 &0 &1\\ 1&\frac 12 &-2 &0\\ 0 &\frac 14&0 &-1 \end{pmatrix} \\ &\leadsto \begin{pmatrix} -1 &\frac 12 & 1 & \frac 12\\ 0 & -\frac 32 &1 &\frac 32\\ 0&1 &-1 &\frac 12\\ 0 &\frac 14&0 &-1 \end{pmatrix} \\ &\leadsto \begin{pmatrix} -1 &\frac 12 & 1 & \frac 12\\ 0&1 &-1 &\frac 12\\ 0 & -3 &2 &3\\ 0 &1&0 &-4 \end{pmatrix} \\ &\leadsto \begin{pmatrix} -1 &\frac 12 & 1 & \frac 12\\ 0&1 &-1 &\frac 12\\ 0 & 0 &-1 &\frac 92\\ 0 &0&1 &-\frac 92 \end{pmatrix} \\ &\leadsto \begin{pmatrix} -1 &\frac 12 & 1 & \frac 12\\ 0&1 &-1 &\frac 12\\ 0 & 0 &-1 &\frac 92\\ 0 &0& 0 &0 \end{pmatrix} \\ \end{align*} So a eigenvector is given by $\varphi = (14, 8,9 , 2)$.

3. The stationary distribution is then given by $\pi := -\frac{\varphi D^{-1}}{\|\varphi D^{-1}\|_1}$. We have $\varphi D^{-1} = (-7, -2, -9, -1)$, so $\|\varphi D^{-1}\|_1 = 19$, that is $\pi = \frac 1{19}(7, 2, 9, 1).$
-
What have you defined lower case d as? – Johnny Byr Oct 18 '12 at 23:50
$\operatorname{Id}$ denotes the Identity. – martini Oct 19 '12 at 5:01