0
$\begingroup$

Say I have 3 states: $A,B,C$, where the transition probabilities are $P_{B\leftarrow A}=P_{C\leftarrow B}=P_{A\leftarrow C}=p$ and $P_{i\leftarrow i}=1-p$ for $i=A,B,C$. All other transition probabilities are zero. The transition matrix is $$P=\begin{pmatrix} 1-p & 0 & p\\ p & 1-p & 0\\ 0 & p & 1-p\\ \end{pmatrix}.$$

Here, $0\leq p\leq 1$.

I find that the stationary distribution for $P$ is$$\rho^*=\begin{pmatrix} 1/3\\ 1/3\\ 1/3\\ \end{pmatrix}.$$

If $p=0$, then any initial distribution vector $\rho_0$ is a stationary distribution; if $p=1$, however, then the components of any initial $\rho_0\neq \rho^*$ will keep hopping/cycling positions. In other words, if I start in say, state $A$ ($\rho_0=(1,0,0)^T$), then the system will never reach a time-independent steady state.

For any $p$ in between 0 and 1, it seems that I will always converge to $\rho^*$.

Two Questions:

1) Can whether or not a state is steady be determined by the eigenvalues of $P$, e.g. whether they are complex or not?

(Eigenvalues of $P$: $\lambda_1=1, \lambda_2=\frac{1}{2}(2 - 3 p - i\sqrt{3}p), \lambda_3=\frac{1}{2}(2 - 3p + i\sqrt{3}p)$).

2) Even though $P$ does not satisfy the detailed balance property, it stills converges to an equilibrium state (depending on the value of $p$). Why is this?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.