I am having trouble understanding how to solve for the state vector at time $t$ for a markov chain using matrix algebra.

I have the following Markov Transition Intensity Matrix, for the states A, N, and D: $\left[ \begin{array}{c c c} -(a+b) & a & b\\ 0 & -g & g\\ 0 & 0 & 0 \end{array} \right]$

I think I have correctly written out the system as two differential equations (because, $D(t) = 1 - (A(t) + N(t))$:

$ \frac{d}{dt} \left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c c} -(a+b) & 0\\ a & -g\\ \end{array} \right] \left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] $

I then get the eigenvalues and eigenvectors as $\lambda_1 = -(a+b)$ with $\nu_1 = \left[ \begin{array}{c} 1\\ \frac{a}{g-a-b}\\\end{array} \right]$, and $\lambda_2= -g$ with $\nu_2 = \left[ \begin{array}{c} 0\\ 1\\\end{array} \right]$.

This is where I think I am going wrong. My understanding is, that I can find the state occupation vector by:

$\left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c} \nu_1 & \nu_2\end{array} \right] \left[ \begin{array}{c c} \exp^{\lambda_1 t} & 0\\ 0 & \exp^{\lambda_2 t}\\ \end{array} \right]\left[ \begin{array}{c} A(0)\\ N(0)\\\end{array} \right]$

$\left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c c} 1 & 0\\\frac{a}{g-a-b} & 1\\ \end{array} \right] \left[ \begin{array}{c c} \exp^{-(a+b)t} & 0\\ 0 & \exp^{-gt}\\ \end{array} \right]\left[ \begin{array}{c} A(0)\\ N(0)\\\end{array} \right]$

$\left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c} A(0)\exp^{-(a+b)t}\\\frac{a}{g-a-b}A(0)\exp^{-(a+b)t} +N(0)\exp^{-gt}\\\end{array} \right]$

But this solution is wrong. It does not agree with the solution obtained using the matrix exponential for the transition intensity matrix. I have been told the solution is $N(t) = \frac{a}{g-a-b}A(0)\exp^{-(a+b)t} +N(0)\exp^{-gt}-\frac{a}{g-a-b}A(0)\exp^{-gt} $

This equation agrees with the matrix exponential. I do not understand how to get this formula based on my reading of solutions to dynamics equations. I was hoping someone could please help me out?

  • $\begingroup$ No. I forgot to add that $D(t) = 1 - ( A(t) + N(t))$. $\endgroup$ – MPahuta Nov 17 '13 at 18:48


$$ \frac{d}{dt} \left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c c} -(a+b) & 0\\ a & -g\\ \end{array} \right] \left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] $$

For this problem, we find the eigenvalues by using $|A - \lambda I| = 0$, and then solve $[A-\lambda_i I]v_i = 0$ to get the eigenvectors. This leads to the eigenvalue / eigenvector results:

  • $\lambda_1 = -(a+b), v_1 = \left(-\dfrac{a+b-g}{a},1\right)$
  • $\lambda_2 = -g, v_2 = (0,1)$

From the eigenvalues / eigenvectors, we can write the Fundamental Matrix using a linear combination as:

$ \phi(t) = \left[ \begin{array}{c} e^{\lambda_1 t}~v_1~~ | ~~e^{\lambda_2 t}~v_2\\\end{array} \right] = \left[ \begin{array}{c c} e^{-(a+b)t} \begin{pmatrix} -\dfrac{(a+b-g)}{a} \\ 1\end{pmatrix} ~~~~ e^{-gt} \begin{pmatrix} 0 \\ 1\end{pmatrix}\\ \end{array} \right] = \begin{bmatrix} -\dfrac{(a+b-g)}{a}e^{-(a+b)t} & 0 \\ e^{-(a+b)t} & e^{-gt} \end{bmatrix}$

Next, we find $\phi(0)$, which is:

$$\phi(0) = \begin{bmatrix} -\dfrac{(a+b-g)}{a} & 0 \\ 1 & 1 \end{bmatrix}$$

Next, find $\phi^{-1}(0)$, which is:

$$\phi^{-1}(0) = \begin{bmatrix} -\dfrac{a}{(a+b-g)} & 0 \\ \dfrac{a}{(a+b-g)} & 1 \end{bmatrix}$$

We can now form the matrix exponential as:

$e^{At} = \phi(t) ~ \phi^{-1}(0) = \begin{bmatrix} e^{-(a+b)t} & ~~~~0 \\ -\dfrac{a}{(g-a-b)}e^{-gt} + \dfrac{a}{g-a-b}e^{-(a+b)t} & ~~~~e^{-gt} \end{bmatrix}$

Now, to form a solution, we multiply the exponential matrix with the initial condition matrix, that is:

$\begin{bmatrix} A(t) \\ N(t) \end{bmatrix} = e^{At}\begin{bmatrix} A(0) \\ N(0) \end{bmatrix} = \begin{bmatrix} e^{-(a+b)t}A(0) \\ \frac{a}{g-a-b}A(0)\exp^{-(a+b)t} -\frac{a}{g-a-b}A(0)\exp^{-gt}+N(0)\exp^{-gt} \end{bmatrix}$

Note, as an alternate approach, you can find the Jordan Normal Form and write:

$$e^{At} = P e^{Jt} P^{-1}$$

where $P$ is a linear combination of the eigenvectors and is used when we can diagonalize the matrix $A$. This of course meant, that we would have found the diagonalized $J = P^{-1} A P$. You should work this out since I provided a solution and then compare to what I wrote. It is maybe an easier approach in this case and I think was the approach you might have been trying to take.

  • $\begingroup$ All that time spent on a great, thorough answer deserves more than my meager upvote, but at least you're on the board! +1 $\endgroup$ – Namaste Nov 18 '13 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.