-1
$\begingroup$

Let $(X_n : n \in \mathbb{N}_0)$ be a Markov chain on finite state space $S = (1,2,3,4,5)$ with the transition probability matrix $P = (p_{ij})_{i,j \in S} $ satisfying

$$\sum_{i \in S}p_{ij} = 1$$

for $j \in S$

Show that $\pi = (1/5, ..., 1/5)$ is a stationary distribution of $(X_n : n \in \mathbb{N}_0)$

I know how to work out $\pi$ for a given P however I am unsure how to "create" or use the satisfying property to directly show what $\pi$ is.

Initial thoughts:

$P$ could be a diagonal matrix with entries of $1/5$ thus $\pi = (1/5, ..., 1/5)$

Thanks in advance.

$\endgroup$
2
$\begingroup$

HINT:

The stationary distribution $\pi$ satisfies the relationship $\pi P=\pi$, which is equivalent to showing $\forall j\in S $:

$$\pi_j = \sum_{i \in S}\pi_i p_{ij}$$

You can also write out $\pi$ explicitly as a row vector where every entry is $\frac{1}{5}$, and $P$ is a matrix with entries $p_{ij}$ in row $i$, column $j$. Expanding this out and using the fact that $\sum_{i \in S} p_{ij} = 1$, you'll see very quickly how you get $\pi P = \pi$

$\endgroup$
  • $\begingroup$ Could I clarify perhaps a simple notation.. Am I to assume that $\sum_{i \in S} p_{ij} = 1$ means each row equals 1? e.g $(p_{11},p_{12},p_{13},p_{14},p_{15}) = 1$ etc for $i = 1,2,3,4,5$ $\endgroup$ – Matt Nov 26 '15 at 5:14
  • $\begingroup$ @Matt It means each column sums to $1$ (j is fixed so it's the column). Try writing it out with a $2x2$ matrix first to see the pattern $\endgroup$ – Brenton Nov 26 '15 at 5:17
  • $\begingroup$ Ah yes, explicitly subbing in $\pi$ and expanding I get $1/5 = 1/5(p_{11},...,p_{51})$ ... $1/5 = 1/5(p_{15},...,p_{55})$. then using the relationship that $(p_{1j}+...+p_{5j}) = 1$ we can see 1/5 = 1/5 for each column. thus $\pi P = \pi$. Thank you. $\endgroup$ – Matt Nov 26 '15 at 5:31
0
$\begingroup$

In general, a measure $\nu:2^S\to [0,\infty)^S$ is said to be invariant under the stochastic kernel $P$ which specifies the transition probabilities of $X$, i.e. $P_{ij} = \mathbb P(X_{n+1}=j\mid X_n=i)$ for $i,j\in S$ if $$\nu = \nu P. $$ Now, since in this case we are also assuming $$\sum_{i\in S}P_{ij} = 1, \ j\in S $$ as well as the usual assumption $$\sum_{j\in S}P_{ij} =1, \ i\in S.$$ So if $\nu(i)=i$ for all $i\in S$, we have that $$\nu(j)\sum_{i\in S}P_{ij}=\nu(j), $$ and thus $\nu$ is an invariant measure for $P$.

If $|S|<\infty$ (the state space is finite, then $$\pi = \frac\nu{|S|}$$ is a stationary distribution for $P$, since $\sum_{i\in S}\pi(S) = 1$ and $\pi P=\pi$. If $S$ is countably infinite, then no stationary distribution exists.

Note in particular that in this case, the stationary distribution is uniform over the state space.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.