Continuous-Time Markov Chains Consider the continuous time MC $X(t)_{t \geq 0}$ with state-space $\{1, 2, 3, 4\}$ and $Q-$matrix 
\begin{equation}
\begin{pmatrix}
-2 & 0 & 0 & 2 \\
1 & -3 & 2 & 0 \\
0 & 2 & -2 & 0 \\
1 & 5 & 2 & -8 
\end{pmatrix}
\end{equation}


*

*(a) Find the invariant distribution for the chain $X(t)_{t \geq 0}$.

*(b) Suppose $X_{0} = 1$. Find for all $t \geq 0$, the probability that $X_{t} = 1$

*(c) Consider the following two processes:
\begin{equation}
Y_{t} = \begin{cases}
          X_{t} & \text{ if }X_{t} \in \{1, 2, 3\} \\
          2 & \text{ if }X_{t} = 4
         \end{cases}
\end{equation}


and 
\begin{equation}
Z_{t} = \begin{cases}
          X_{t} & \text{ if }X_{t} \in \{1, 2, 3\} \\
          1 & \text{ if }X_{t} = 4
         \end{cases}
\end{equation}
Determine which, if any, of the processes $Y(t)_{t \geq 0}$ and $Z(t)_{t \geq 0}$, is a Markov chain.
I have solved part (a) and (b) and I found the invariant distribution to be $(1/5,7/20,2/5,1/20)$ and $P_{11}(t) = \exp(-2t)$.
Can anyone give me a hint on how to think in the last part? How can I link $X(t)$ and $Y(t)$ and $Z(t)$?
 A: It may help to think in terms of the associated discrete time processes, which I'll denote $x_n, y_n, z_n$; the transition matrix for $x_n$ is given by:
$$
P_x =
\begin{pmatrix}
0 & 0 & 0 & 1 \\
1/3 & 0 & 2/3 & 0 \\
0 & 1 & 0 & 0 \\
1/8 & 5/8 & 1/4 & 0 
\end{pmatrix}
$$
The processes $y_n, \, z_n$ have the same transition rules as the continuous time versions. Consider first $y_n$
\begin{equation}
y_n = \begin{cases}
          x_n & \text{ if }x_{n} \in \{1, 2, 3\} \\
          2 & \text{ if }x_{n} = 4
         \end{cases}
\end{equation}
and 
\begin{equation}
z_{n} = \begin{cases}
          x_{n} & \text{ if }x_{n} \in \{1, 2, 3\} \\
          1 & \text{ if }x_{n} = 4
         \end{cases}
\end{equation}
Note that from the matrix $P_x$ we can see that the special condition in the definition of $y_n$ can only occur if $z_{n-1} = 1$ (since the probability of jumping to state $4$ is $0$ from the other sites). Moreover, if $x_n = 1$, then it almost surely jumps to site $4$. As a result, the process $P_y$ can be described with a transition matrix
$$ P_y = 
\begin{pmatrix}
0 & 1 & 0 \\
1/3 & 0 & 2/3 \\
0 & 1 & 0
\end{pmatrix}.
$$
So we see that there is a discrete time Markov chain which is described by the dynamic of $x_n$. So can we then derive a continuous time chain? To do this, we would need to determine jump rates out of each site $\{1,2,3\}$. 
The jump rate from sites $1$ and $3$ are the same as for $X_t$. The problem is site $2$: if $Y_t$ at site $2$ from site $3$, then we are following the normal dynamic of $X_t$, and so the holding time of $Y_t$ at $2$ will be the holding time of $X_t$ at $2$, which is $1/3$. However, if $Y_t$ arrives at $2$ because $X_t = 4$, then the holding time is $1/8$ (i.e. the holding time of $X_t$ at $4$).
As a result, we cannot ascribe a single holding time to site $2$, and therefore we cannot determine a $Q$-matrix for the process. As such it is not Markov.
An alternate statement would be that the holding time of $Y_t$ at site $2$ is dependent on which site $Y_t$ arrived at $2$ from; you can then check this against the definition of the Markov property for continuous time chains, and see that this forms a contradiction.
Considering instead the process $Z_t$; again the discrete analogue has a transition matrix, this time given by:
$$ P_z = 
\begin{pmatrix}
1 & 0 & 0 \\
1/3 & 0 & 2/3 \\
0 & 1 & 0
\end{pmatrix}.
$$
In this case, we see that site $1$ is now an absorbing state for the Markov chain: i.e. once $Z_n = 1$, then almost surely $Z_m = 1$ for all $m \geq 1$. Whereas before, we could not uniquely determine a jump rate of $Y_t$ out of site $2$, in this case we know that the jump rate of $Z_t$ out of state $1$ must be $0$ (it does not leave). As such we can write down a $Q$-matrix:
$$ Q_Z = 
\begin{pmatrix}
0 & 0 & 0 \\
1 & -3 & 2 \\
0 & 2 & -2
\end{pmatrix},
$$
where the first row being equal to $0$ says exactly that state $1$ is absorbing.
