Accumulating errors with different rates for each state I'm struggling with this question. It's like a Nonhomogeneous Poisson Process applied to Markov Chains. 
I have a Markov Chain composed of three states (1,2,3) as follows. 
\begin{bmatrix}0.5&0.3&0.2\\0&0.6&0.4\\0&0&1\end{bmatrix}
The number of error messages I get in each state is a Poisson Process. The mean of the number of error messages I get per minute is 0, 1, and 2 for states 1, 2, 3, respectively.
What is the probability that I will have accumulated 4 error messages when the 5th minute finishes given that the initial state is 1?
Or in general, what is the probability that I have a total of k messages at the end of nth minute given the initial state is s? How can I generalize it for large n given the initial state s? 
 A: This is a Markov-modulated Poisson process. Let $\{X_n:n=0,1,\ldots\}$ be the underlying Markov chain and $N(t)$ the arrival process. Then $N(t) = N_1(t)\mathsf 1_{\{2\}}(X_n) + N_2(t)\mathsf 1_{\{3\}}(X_n)$ where $N_1(t)$ is a Poisson process with rate $1$ and $N_2(t)$ a Poisson process with rate $2$.
Assume $X_0=1$. Let $\tau_i = \inf\{n>0:X_n=i\}$, $i=2,3$. Then
$$
\mathbb P(\tau_2=i) =\frac3{10}\left(\frac12\right)^{i-1},\ i=1,2,\ldots
$$
and
$$
\mathbb P(\tau_3=j) = \begin{cases}
\frac15,& j=1\\
\frac{11}{50},& j=2\\
\frac{91}{500},& j=3\\
\frac{851}{5000},& j=4
\end{cases}
$$
We have
\begin{align}
\mathbb P(N(5)=m) &= \mathbb P(\tau_3=1)\mathbb P(N_2(5)-N_2(1)=m)+\sum_{i=1}^4\mathbb P(\tau_2=i)\mathbb P(\tau_3\geqslant 5)\mathbb P(N_1(5)-N_1(i)=m)+\sum_{i=1}^4\sum_{j=i+1}^4\mathbb P(\tau_2=i)\mathbb P(\tau_3=j)\sum_{m_1=0}^m \mathbb P(N_1(j)-N_1(i)=m_1)\sum_{m_2=0}^{m-m_1}\mathbb P(N_2(5)-N_2(j)=m_2).
\end{align}
Unfortunately, lack of a closed form for $\mathbb P(\tau_3=j)$ makes this very tedious to evaluate. But I believe the idea is correct.
