0
$\begingroup$

Problem:

Does there exist an invariant probability vector $\bar{\pi}=(\pi(i))_{i \in S}$, where $S$ is the state space of the Markov chain, such that $\pi(1,2)=0.2$?

I have already calculated that, given that the chain starts in state $(1,2)$, the invariant probability for that specific recurrent class, is $$\bar{\pi}_C=\Big[\pi(1,3) \quad \pi(2,2) \quad \pi(1,2) \quad \pi(2,3) \quad \pi(0,3)\Big] =\Big[\frac{2}{7} \quad \frac{1}{7} \quad \frac{1}{7} \quad \frac{2}{7} \quad \frac{1}{7} \Big]$$

My attempt:

My attempt:

So my first answer was no, because for the recurrent class, $\pi(1,2)=\frac{1}{7}$. But I forgot to account for if we started in the transient class. And I'm not quite sure how to add that into the equation.

$\endgroup$
1
$\begingroup$

Intuitively, no matter the starting position, all the probability eventually 'drains' into either the class $\{(1,0), (2,0)\}$ or the class $\{(0,3),(1,3),(2,3),(1,2),(2,2)\}$. So any invariant probability can only be non-zero here.

Inside the second of these classes, the ratio between each of the states is fixed (by aperiodicity), and so the probability contained in this class must be seven times $\pi(1,2)$. Can you see why the answer is therefore no?

$\endgroup$
  • $\begingroup$ Yes I think so. But what about the other class, $\{(1,0),(2,0)\}$? Can't there be a distribution that takes both recurrent classes into account? $\endgroup$ – KSHMR Dec 20 '17 at 17:46
  • $\begingroup$ There certainly can be. Suppose there is, what happens to the total probability contained in the second class? Perhaps I wasn't clear enough in my answer. $\endgroup$ – B. Mehta Dec 20 '17 at 17:50
  • $\begingroup$ Sorry but do you think you could expand a little bit on your answer? $\endgroup$ – KSHMR Dec 20 '17 at 17:56
  • $\begingroup$ If there's some probability in the first class, then the total probability in the second class must be less than 1. But, the probability of $\pi(1,2)$ must be a seventh of the total probability in the second class. $\endgroup$ – B. Mehta Dec 20 '17 at 17:57
  • $\begingroup$ I was wondering, why does aperiodicity imply that the ratio of invariant probabilities in the recurrent class is fixed? $\endgroup$ – KSHMR Dec 20 '17 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.