1
$\begingroup$

If $X$ is a discrete Markov chain with state space $S=\{1,2\}$ and transition matrix

\begin{equation*} P=\begin{pmatrix} 1-a& a\\ b& 1-b \end{pmatrix}. \end{equation*}

I must answer the question "Classify the states of the chain". What is meant by this? Must I say if the states are recurrent or transient? And if so, which one is it?

$\endgroup$
  • $\begingroup$ Without seeing the context, I guess I would assume they want to know recurrent or transient. I'm assuming you know the definition of those terms; what do you think the states are? $\endgroup$ – Brian Tung May 5 '15 at 20:29
  • $\begingroup$ There is no further context given, and I think recurrent, but I'm not sure $\endgroup$ – user235238 May 5 '15 at 20:31
  • $\begingroup$ Yes, under fairly broad assumptions ($0 < a, b \leq 1$), the states are recurrent. If both $a = b = 1$, the states are periodic. If only one of them equals $1$, then the states are merely recurrent. $\endgroup$ – Brian Tung May 5 '15 at 20:36
1
$\begingroup$

If $0<a,b\leqslant1$ then $P_{ij}+P^2_{ij}>0$ for all $i,j$ so the Markov chain is (positive) recurrent. You can verify this by computing $\mathbb E_i[\tau_i]$ where $$\tau_i = \inf\{n>0 : X_n=0\},$$ and $\mathbb E_i[\cdot]$ denotes conditioning on $X_0=i$.

If $a=0$ (resp. $b=0$) then state $1$ (resp. state $2$) is absorbing, and therefore transient.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy