Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Again a probability exercise:

Let $X=U \cup V$ be the finite state space of a Markov chain, where $U$ and $V$ are disjoint subsets of $X$ and $p_{ij}=0$ if both $i,j \in U$ or both $i,j \in V$. Otherwise, they are positive. Find a stationary distribution?

I'm assuming that it has multiple distributions but I wrote down the system that I'm trying to solve and either I'm really tired or I should be thinking of smth else: if $\pi$ is the stationary distribution then I'm trying to find we have $\pi_{j} = \sum_{i \in V} {\pi_{i} p_{ij}}$ for all $j \in U$ and also $\pi_{j} = \sum_{i \in U} {\pi_{i} p_{ij}}$ for all $j \in V$... Can someone please help again?


share|cite|improve this question
Anna: Why do you abstain from accepting the answer below? I doubt anything meaningful can be added to it... – Did Apr 12 '12 at 7:09

The wording of this exercise seems a bit strange to me, since every finite state space Markov chain has a stationary distribution.

Added: Let $P$ be the transition matrix of your chain. Any stationary distribution $\pi$ for $P$ is also stationary for $P^2$. But under $P^2$, the sets $U$ and $V$ are disjoint, recurrent communicating classes with unique stationary distributions $\pi_U$ and $\pi_V$ (of lengths $|U|$ and $|V|$).

Let $\tilde\pi_U$ and $\tilde\pi_V$ be vectors of length $|X|$ obtained by padding $\pi_U$ and $\pi_V$ with zeros in the appropriate places. Then the set of all stationary distributions for $P^2$ is $$\{\alpha\tilde\pi_U+(1-\alpha)\tilde\pi_V,\, 0\leq \alpha\leq 1\}.$$

The only one of these that is also stationary for $P$ is ${1\over 2}\tilde\pi_U+ {1\over 2}\tilde\pi_V$.

share|cite|improve this answer
It should have read "find a stationary distribution" – Anna Mar 12 '12 at 16:51
I'm not sure this is right. Maybe we just disagree on the definition of stationary; I was told that as a matter of fact this Markov chain does not have a stationary distribution.. – Anna Mar 12 '12 at 22:10
@Anna Maybe you should add the definition of "stationary distribution" to your question. Usually it means a row vector $\pi$ of probabilities that satisfies $\pi=\pi P$. – Byron Schmuland Mar 12 '12 at 22:46

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.