# Markov Chain Solution Eigenvalue

I am having trouble understanding how to solve for the state vector at time $t$ for a markov chain using matrix algebra.

I have the following Markov Transition Intensity Matrix, for the states A, N, and D: $\left[ \begin{array}{c c c} -(a+b) & a & b\\ 0 & -g & g\\ 0 & 0 & 0 \end{array} \right]$

I think I have correctly written out the system as two differential equations (because, $D(t) = 1 - (A(t) + N(t))$:

$\frac{d}{dt} \left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c c} -(a+b) & 0\\ a & -g\\ \end{array} \right] \left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right]$

I then get the eigenvalues and eigenvectors as $\lambda_1 = -(a+b)$ with $\nu_1 = \left[ \begin{array}{c} 1\\ \frac{a}{g-a-b}\\\end{array} \right]$, and $\lambda_2= -g$ with $\nu_2 = \left[ \begin{array}{c} 0\\ 1\\\end{array} \right]$.

This is where I think I am going wrong. My understanding is, that I can find the state occupation vector by:

$\left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c} \nu_1 & \nu_2\end{array} \right] \left[ \begin{array}{c c} \exp^{\lambda_1 t} & 0\\ 0 & \exp^{\lambda_2 t}\\ \end{array} \right]\left[ \begin{array}{c} A(0)\\ N(0)\\\end{array} \right]$

$\left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c c} 1 & 0\\\frac{a}{g-a-b} & 1\\ \end{array} \right] \left[ \begin{array}{c c} \exp^{-(a+b)t} & 0\\ 0 & \exp^{-gt}\\ \end{array} \right]\left[ \begin{array}{c} A(0)\\ N(0)\\\end{array} \right]$

$\left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c} A(0)\exp^{-(a+b)t}\\\frac{a}{g-a-b}A(0)\exp^{-(a+b)t} +N(0)\exp^{-gt}\\\end{array} \right]$

But this solution is wrong. It does not agree with the solution obtained using the matrix exponential for the transition intensity matrix. I have been told the solution is $N(t) = \frac{a}{g-a-b}A(0)\exp^{-(a+b)t} +N(0)\exp^{-gt}-\frac{a}{g-a-b}A(0)\exp^{-gt}$

This equation agrees with the matrix exponential. I do not understand how to get this formula based on my reading of solutions to dynamics equations. I was hoping someone could please help me out?

• No. I forgot to add that $D(t) = 1 - ( A(t) + N(t))$. – MPahuta Nov 17 '13 at 18:48

Given:

$$\frac{d}{dt} \left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right] = \left[ \begin{array}{c c} -(a+b) & 0\\ a & -g\\ \end{array} \right] \left[ \begin{array}{c} A(t)\\ N(t)\\\end{array} \right]$$

For this problem, we find the eigenvalues by using $|A - \lambda I| = 0$, and then solve $[A-\lambda_i I]v_i = 0$ to get the eigenvectors. This leads to the eigenvalue / eigenvector results:

• $\lambda_1 = -(a+b), v_1 = \left(-\dfrac{a+b-g}{a},1\right)$
• $\lambda_2 = -g, v_2 = (0,1)$

From the eigenvalues / eigenvectors, we can write the Fundamental Matrix using a linear combination as:

$\phi(t) = \left[ \begin{array}{c} e^{\lambda_1 t}~v_1~~ | ~~e^{\lambda_2 t}~v_2\\\end{array} \right] = \left[ \begin{array}{c c} e^{-(a+b)t} \begin{pmatrix} -\dfrac{(a+b-g)}{a} \\ 1\end{pmatrix} ~~~~ e^{-gt} \begin{pmatrix} 0 \\ 1\end{pmatrix}\\ \end{array} \right] = \begin{bmatrix} -\dfrac{(a+b-g)}{a}e^{-(a+b)t} & 0 \\ e^{-(a+b)t} & e^{-gt} \end{bmatrix}$

Next, we find $\phi(0)$, which is:

$$\phi(0) = \begin{bmatrix} -\dfrac{(a+b-g)}{a} & 0 \\ 1 & 1 \end{bmatrix}$$

Next, find $\phi^{-1}(0)$, which is:

$$\phi^{-1}(0) = \begin{bmatrix} -\dfrac{a}{(a+b-g)} & 0 \\ \dfrac{a}{(a+b-g)} & 1 \end{bmatrix}$$

We can now form the matrix exponential as:

$e^{At} = \phi(t) ~ \phi^{-1}(0) = \begin{bmatrix} e^{-(a+b)t} & ~~~~0 \\ -\dfrac{a}{(g-a-b)}e^{-gt} + \dfrac{a}{g-a-b}e^{-(a+b)t} & ~~~~e^{-gt} \end{bmatrix}$

Now, to form a solution, we multiply the exponential matrix with the initial condition matrix, that is:

$\begin{bmatrix} A(t) \\ N(t) \end{bmatrix} = e^{At}\begin{bmatrix} A(0) \\ N(0) \end{bmatrix} = \begin{bmatrix} e^{-(a+b)t}A(0) \\ \frac{a}{g-a-b}A(0)\exp^{-(a+b)t} -\frac{a}{g-a-b}A(0)\exp^{-gt}+N(0)\exp^{-gt} \end{bmatrix}$

Note, as an alternate approach, you can find the Jordan Normal Form and write:

$$e^{At} = P e^{Jt} P^{-1}$$

where $P$ is a linear combination of the eigenvectors and is used when we can diagonalize the matrix $A$. This of course meant, that we would have found the diagonalized $J = P^{-1} A P$. You should work this out since I provided a solution and then compare to what I wrote. It is maybe an easier approach in this case and I think was the approach you might have been trying to take.

• All that time spent on a great, thorough answer deserves more than my meager upvote, but at least you're on the board! +1 – Namaste Nov 18 '13 at 14:30