Markov chain state time I have a question regarding markov chains in continuous time. 
I have a birth/death process(for every step we go 1 up & or 1 down), queue with 4 states. Customers arrive with the intensity of lambda per hour and can be seen as poisson distributed. The time to serve each customer can be seen as exponentially distributed and independent.
We start at time = 0 and state = 0. We are going to do 100 steps. 
State 0 => There is no customers. (Arrival rate = 2, Departure rate = 0)
State 1 => There is one customer being served. (Arrival rate = 2, Departure rate = 10)
State 2 => There is one customer being served + 1 in queue. (Arrival rate = 2, Departure rate = 10)
State 3 => There is one customer being served + 2 in queue. (Arrival rate = 0, Departure rate = 10)
Givens: 
Arrival rate = 2. 
Depature rate = 10. 
Is there a formula to calculate the total time spent in each state? 
 A: Under the assumption that arriving customers are turned away when there are three customers in the system, this is a $M/M/1/3$ queue, that is, a single-server queueing system with arrival process Poisson, i.i.d. exponentially distributed service times, and a finite buffer size of $3$. Let $X(t)$ be the number of customers in the system at time $t$. Then $X=\{X(t):t\in\mathbb R_+\}$ is a continuous-time Markov chain with state space $S = \{0,1,2,3\}$ and transition rates 
$$
q_{ij} =
\begin{cases}
2,& j=i+1\\
10,& j=i-1\\
0,& \text{otherwise}.
\end{cases}
$$
Let $P(t)$ be the matrix with entries $$[P(t)_{ij}] = \mathbb P(X(t) = j\mid X(0) = i).$$
Then $P$ satisfies the forward Kolmogorov equation $$P'(t) = P(t)Q(t), $$ with solution given by the matrix exponential $$P(t) = e^{tQ}.$$ (This may be verified by the Chapman-Kolmogorov equation $P(s+t) = P(s)P(t)$.) Since each row of $P(t)$ sums to $1$ (as it is a stochastic matrix), it follows that each row of $Q$ sums to $0$. The nondiagonal entries of $Q$ are precisely the transition rates $q_{ij}$, so the diagonal entries satisfy $$q_{ii} = -\sum_{j\in S\setminus\{i\}}q_{ij}. $$
Hence,
$$
Q = \begin{pmatrix}
-2 & 2 & 0 & 0\\
10 & -12 & 2 & 0\\
0 & 10 & -12 & 2\\
0 & 0 & 10 & -10.
\end{pmatrix}
$$
If $\pi$ is a probability distribution on $S$, let $\{X_\pi(t):t\in\mathbb R_+\}$ denote the process $X$ conditioned on $X(0)\sim\pi$. If $X_\pi(t)$ is a stationary process, that is, $\{X_\pi(s+t):t\in\mathbb R_+\}\stackrel d=\{X_\pi(t):t\in\mathbb R_+\}$ for all $s\geqslant 0$, then $\pi$ is a stationary distribution. From the balance equations $$\sum_{j\in S\setminus\{ i\}} \pi_{i}q_{ij} = \sum_{j\in S\setminus\{ i\}} \pi_{j}q_{ji},$$ it can be verified that $\pi$ is a stationary distribution iff $\pi Q=0$. In this example, $\pi Q=0$ is the system of linear equations
\begin{align}
-2\pi_0 + 10\pi_1 &= 0\\
2\pi_0 -12\pi_1 + 10\pi_2 &= 0\\
2\pi_1 -12\pi_2 + 10\pi_3 &= 0\\
2\pi_2 -10\pi_3 &= 0
\end{align}.
These equations, along with $\sum_{i\in S}\pi_i = 1$ (since $\pi$ is a probability distribution), yield
$$\pi = \begin{pmatrix}\dfrac{125}{156}& \dfrac{25}{156}& \dfrac{5}{156}& \dfrac1{156} \end{pmatrix}. $$
The fraction of time spent in each state satisfies
$$\lim_{T\to\infty}\frac1T \int_0^T \mathsf 1_{\{X(t)=i\}}\,\mathsf dt = \pi(i).$$
This serves as a good approximation for the fraction of time spent in a state for large $t$, but computing the exact value of $\int_0^T \mathsf 1_{\{X(t)=i\}}\,\mathsf dt$ for a given $T$ is non-trivial and (especially for larger problems) is generally done by simulation.
