# What does this question about classifying the states of this Markov chain mean?

If $X$ is a discrete Markov chain with state space $S=\{1,2\}$ and transition matrix

\begin{equation*} P=\begin{pmatrix} 1-a& a\\ b& 1-b \end{pmatrix}. \end{equation*}

I must answer the question "Classify the states of the chain". What is meant by this? Must I say if the states are recurrent or transient? And if so, which one is it?

• Without seeing the context, I guess I would assume they want to know recurrent or transient. I'm assuming you know the definition of those terms; what do you think the states are? – Brian Tung May 5 '15 at 20:29
• There is no further context given, and I think recurrent, but I'm not sure – user235238 May 5 '15 at 20:31
• Yes, under fairly broad assumptions ($0 < a, b \leq 1$), the states are recurrent. If both $a = b = 1$, the states are periodic. If only one of them equals $1$, then the states are merely recurrent. – Brian Tung May 5 '15 at 20:36

If $0<a,b\leqslant1$ then $P_{ij}+P^2_{ij}>0$ for all $i,j$ so the Markov chain is (positive) recurrent. You can verify this by computing $\mathbb E_i[\tau_i]$ where $$\tau_i = \inf\{n>0 : X_n=0\},$$ and $\mathbb E_i[\cdot]$ denotes conditioning on $X_0=i$.
If $a=0$ (resp. $b=0$) then state $1$ (resp. state $2$) is absorbing, and therefore transient.