I'm currently trying to solve this problem about Markov chains.

Let a Markov chain have the transition probability matrix:

$$ P = \begin{bmatrix} 1 - \alpha & \alpha \\ \beta & 1 - \beta \end{bmatrix} $$

where $|1 - \alpha - \beta| \lt 1$.

1) Find the $n$-step transition matrix $P^n$ as $n \rightarrow \infty$.

2) Show that for any initial probability the Markov chain has the same probability distribution as $n \rightarrow \infty$.

For the first problem, I was able to find that the transition matrix is:

$$ P^n = \begin{bmatrix} \dfrac{\alpha + \beta(1 - \alpha - \beta)^n}{\alpha + \beta} & \dfrac{\alpha - \alpha(1 - \alpha - \beta)^n}{\alpha + \beta} \\ \dfrac{\beta - \beta(1 - \alpha - \beta)^n}{\alpha + \beta} & \dfrac{\beta + \alpha(1 - \alpha - \beta)^n}{\alpha + \beta} \end{bmatrix} $$

and so for $n \rightarrow \infty$ we have:

$$ P^{n \rightarrow \infty} = \begin{bmatrix} \dfrac{\alpha}{\alpha + \beta} & \dfrac{\alpha}{\alpha + \beta} \\ \dfrac{\beta}{\alpha + \beta} & \dfrac{\beta}{\alpha + \beta} \end{bmatrix} $$

If I'm incorrect, please let me know.

What's confusing me is the second problem. To be completely frank, I don't recall learning this in the course I'm taking and I'm having trouble finding resources online, which led me to ask a question here. How would I solve this problem? I'm having trouble from where to begin.

Any tips or feedback are appreciated. Thanks in advance!

  • $\begingroup$ Let $\pi$ be the initial distribution, let $Q$ be your limit matrix. You have to show that $\pi Q$ does not depend on $\pi$. $\endgroup$ – kimchi lover Dec 14 '19 at 15:29

Let $Q = \lim_{n\to\infty} P^n$. For any initial distribution $\alpha=(\alpha_0,\alpha_1)$, we have $$ \alpha Q=\begin{pmatrix}\alpha_0&\alpha_1\end{pmatrix}\left( \large\begin{array}{cc} \frac{\alpha }{\alpha +\beta } & \frac{\alpha }{\alpha +\beta } \\ \frac{\beta }{\alpha +\beta } & \frac{\beta }{\alpha +\beta } \\ \end{array} \right) = \left( \Large\begin{array}{cc} \frac{\alpha \alpha _0}{\alpha +\beta }+\frac{\beta \alpha _1}{\alpha +\beta } & \frac{\alpha \alpha _0}{\alpha +\beta }+\frac{\beta \alpha _1}{\alpha +\beta } \\ \end{array} \right) $$ which shows that $\lim_{n\to\infty}\mathbb P(X_n=j\mid X_0=i)$ is independent of $i$.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.