0
$\begingroup$

Given transition matrix probability $$ \begin{bmatrix} 1&0&0&0\\ 0.1&0.6&0.1&0.2\\ 0.2&0.3&0.4&0.1\\ 0&0&0&1 \end{bmatrix} $$ with state = $\{0,1,2,3\}$. Start from state $1$, find probability of markov chain ended in state $0$.

I have found state transition diagram. But I don't know to find probability of Markov chain ended in state $0$. What formula can be used to answer this problem? Can anyone help me?

$\endgroup$
3
$\begingroup$

Note that $X_n$ of the given transition probability is eventually absorbed at $\{0,3\}$ with probability $1$. For $i=1,2$, let $p_i=\Bbb P(X_n\text{ is absorbed at }0\mid X_0=i)$. Then we can see that $$\begin{align*}\require{cancel} p_1&=\sum_{k=0}^3\Bbb P(X_n\text{ is absorbed at }0, X_1=k\mid X_0=1) \\&=\sum_{k=0}^3\Bbb P(X_n\text{ is absorbed at }0\mid X_1=k\cancel{ X_0=1})\Bbb P(X_1=k\mid X_0=1) \\&=1\cdot (0.1)+p_1\cdot (0.6)+p_2\cdot (0.1)+0\cdot (0.2). \end{align*}$$ In the same way, $$ p_2 = (0.2)+p_1\cdot (0.3)+p_2\cdot (0.4). $$ holds. Solving those 2 equations, we obtain $$ p_1= \frac8{21}, \ \ \ p_2=\frac{11}{21}. $$

Note: The vector $\pi_0=(1,p_1,p_2,0)$ of probability being absorbed at $0$ satisfies the equilibrium equation $$ P\pi_0=\pi_0 $$where $P$ is the transition matrix, because $\pi_0 = \lim_{n\to\infty} P^n (0,1,0,0).$

$\endgroup$
  • $\begingroup$ So, the answer is two? 8/21 and 11/21? i=1,2 because it is transient states? $\endgroup$ – Ongky Denny Wijaya Mar 3 at 13:41
  • $\begingroup$ What you want is the probability of absorption at $0$ starting from $\color{red}1$. So the answer is $p_1=\frac8{21}$. $p_2$ is only need to solve the equation. $\endgroup$ – Song Mar 3 at 13:42
  • $\begingroup$ Ok, thanks for your answer. I will try to understand it. $\endgroup$ – Ongky Denny Wijaya Mar 3 at 13:45
  • 1
    $\begingroup$ I hope this will help. $\endgroup$ – Song Mar 3 at 13:50
0
$\begingroup$

You can also use matrix algebra to solve for the markov chain. I do not know if you have been taught this method. But you can find the material in Wikepedia.

solution

$\endgroup$
  • $\begingroup$ Thanks for your answer, I will try to understand it $\endgroup$ – Ongky Denny Wijaya Mar 3 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.