2
$\begingroup$

Let's define two stats Markov chain:
$P=\left( \begin{array}{ccc} 0 & 1 \\ 1/2 &1/2 \end{array} \right)$
Then, stationary distribution is:
$$sP=s$$ $$\left(0s_1+\frac12s_2, s_1+\frac12s_2\right)=\left(\frac12s_2, s_1+\frac12 s_2 \right) = (s_1, s_2)$$
And, $\frac12 s_2+ s_1+\frac12s_2 = 1$
So have exactly one solution, $s_1=\frac13, s_2=\frac23$.
Our chain has exactly one stationary distribution.

Is it ok computed ?
My mainly question is- how to compute period for state $1$ and state $2$ ?
From definition period is $o(s) =\gcd\left\{n : p_{ss}(n) > 0\right\}$.

$\endgroup$
4
  • 2
    $\begingroup$ Your computation is correct. Since the Markov chain is irreducible and $p_{22}>0$, it's clear that the period of both states is $1$ (that is, the chain is aperiodic). $\endgroup$
    – Math1000
    Aug 31 '16 at 9:55
  • $\begingroup$ $p_{1,2}(1) = 1$, $p_{2,2}(1) =1/2$, $p_{2,2} = 1/2 + 1/2p_{1,2}=1/2+1/2 = 1$, $p_{1,1}(1) = 0$, $p_{1,1}(2) = 1\cdot p_{2,1}$ Lets try to find $o(1)=\gcd\{n:p_{1,1}(n) > 0\} = \gcd\{2,3,4,5,....\}=1$ $o(2) = \{1,2,3...\} = 1$. Tell me please, these compuations are ok ? $\endgroup$
    – user343207
    Aug 31 '16 at 10:24
  • 2
    $\begingroup$ That is correct, but it should be intuitively clear why the chain is aperiodic, from the definition, without need for explicit computation. $\endgroup$
    – Math1000
    Aug 31 '16 at 10:27
  • $\begingroup$ Yes, I understand it. It is about your first comment. Thanks $\endgroup$
    – user343207
    Aug 31 '16 at 10:37
2
$\begingroup$

Since the Markov chain is irreducible and aperiodic with a finite state space, we may also compute the stationary distribution $s$ by computing $\lim_{n\to\infty} P^n$ and taking any of its rows. Since $P^n$ is a stochastic matrix (i.e. the rows sum to $1$), it is clear that $(1,1)$ is an eigenvector with associated eigenvalue $1$. The sum of the eigenvalues of a matrix is equal to its trace (the sum of the diagonal elements), so the other eigenvalue is $-\frac12$. Solving $\left(P-\left(\frac12\right)I\right)x=0$ yields $(2,-1)$ as a solution. So we may write $P=ADA^{-1}$ where $$A = \begin{pmatrix}1&2\\1&-1\end{pmatrix},\quad D = \begin{pmatrix}1&0\\0&-\frac12\end{pmatrix}. $$ By induction we see that $P^n = AD^nA^{-1}$, and since $A^{-1}=\frac13 A$ we have $$P^n = \frac13\begin{pmatrix}1&2\\1&-1\end{pmatrix} \begin{pmatrix}1&0\\0&\left(-\frac12\right)^n\end{pmatrix}\begin{pmatrix}1&2\\1&-1\end{pmatrix}=\frac13\begin{pmatrix}1 - \left(\frac12\right)^{n-1}& 2\left(1-\left(\frac12\right)^n\right) \\ 1- \left(\frac12\right)^n & 2\left(1+\left(\frac12\right)^n\right)\end{pmatrix}. $$ It follows that $$\lim_{n\to\infty} P^n = \begin{pmatrix}\frac13&\frac23\\\frac13&\frac23, \end{pmatrix} $$ and so the stationary distribution is given by $$s = \left(\frac13,\frac23\right).$$

$\endgroup$
6
  • 1
    $\begingroup$ Here, you are using ergodic theorem, yeah ? $\endgroup$
    – user343207
    Aug 31 '16 at 17:49
  • 1
    $\begingroup$ Yes; note that by definition, a state is ergodic iff it is aperiodic and positive recurrent. When the state space is finite, irreducibility implies that every state is positive recurrent. $\endgroup$
    – Math1000
    Aug 31 '16 at 18:19
  • $\begingroup$ Why did you say ergodic state ? I thought that ergodicity is about chain (not single state). Is it possible that chain (that contains periodic states) has exactly one stationary distribution ? $\endgroup$
    – user343207
    Aug 31 '16 at 18:31
  • 1
    $\begingroup$ If all states are ergodic, then the chain is ergodic. But it is certainly possible for some states to be ergodic and others not - consider a Markov chain with two states and exactly one of them absorbing. $\endgroup$
    – Math1000
    Aug 31 '16 at 18:59
  • 1
    $\begingroup$ Yes; consider the transition matrix $$P=\begin{pmatrix}0&1\\1&0\end{pmatrix}.$$ Then $\pi P=\pi$ implies $\pi_1=\pi_2$, so $\left(\frac12,\frac12\right)$ is the only stationary distribution. But clearly this Markov chain is periodic with period $2$. $\endgroup$
    – Math1000
    Aug 31 '16 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy