Say I have 3 states: $A,B,C$, where the transition probabilities are $P_{B\leftarrow A}=P_{C\leftarrow B}=P_{A\leftarrow C}=p$ and $P_{i\leftarrow i}=1-p$ for $i=A,B,C$. All other transition probabilities are zero. The transition matrix is $$P=\begin{pmatrix} 1-p & 0 & p\\ p & 1-p & 0\\ 0 & p & 1-p\\ \end{pmatrix}.$$

Here, $0\leq p\leq 1$.

I find that the stationary distribution for $P$ is$$\rho^*=\begin{pmatrix} 1/3\\ 1/3\\ 1/3\\ \end{pmatrix}.$$

If $p=0$, then any initial distribution vector $\rho_0$ is a stationary distribution; if $p=1$, however, then the components of any initial $\rho_0\neq \rho^*$ will keep hopping/cycling positions. In other words, if I start in say, state $A$ ($\rho_0=(1,0,0)^T$), then the system will never reach a time-independent steady state.

For any $p$ in between 0 and 1, it seems that I will always converge to $\rho^*$.

Two Questions:

1) Can whether or not a state is steady be determined by the eigenvalues of $P$, e.g. whether they are complex or not?

(Eigenvalues of $P$: $\lambda_1=1, \lambda_2=\frac{1}{2}(2 - 3 p - i\sqrt{3}p), \lambda_3=\frac{1}{2}(2 - 3p + i\sqrt{3}p)$).

2) Even though $P$ does not satisfy the detailed balance property, it stills converges to an equilibrium state (depending on the value of $p$). Why is this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.