Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two Markov chains $X_n$, $Y_n$ with the same transision matrix P, which is non-periodic and non separable. The initial distribution is $\pi_x = \frac{1}{3}[1,1,1]$ and $\pi_y$ is unknown.

Define the stopping time: $T = \inf\{n\geq 0 : X_n=Y_n\}$ , I need to find $P(T>n)$

Now, I know that the first distribution makes $X_n$ invariant, and that from some $n$ they will be of the same distribution... But I don't really know how to approach the problem.

Appreciate and help.

share|cite|improve this question
are $X_n$ and $Y_n$ independent, and what do you know about $P$ that tells you that $\pi$ is its stationary measure ? – mike Jun 7 '12 at 14:59
Any luck with my answer below? – Did Jul 12 '12 at 10:41
Bis repetita... – Did Jul 23 '12 at 10:29

For every $a\geqslant0$ and $b\geqslant0$ such that $0\lt a+b\leqslant1$, the transition matrix $$ P=\begin{pmatrix}1-a-b&a&b\\ b&1-a-b&a\\ a& b&1-a-b\end{pmatrix} $$ is irreducible and aperiodic with uniform stationary distribution, but, even assuming that $(X_n)$ and $(Y_n)$ are independent, the distribution of $T$ depends very much on $(a,b)$. You can try to compute the cases (i) $a=b=\frac12$, (ii) $a=b=\frac13$ and (iii) $b=0$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.