Given the probability matrix $P$ with states $s_1...s_5$ where $s_1$ and $s_5$ are absorbing

$$ P = \left[ \begin{matrix} 1 & 0.7 & 0 & 0 & 0 \\ 0 & 0 & 0.5 & 0 & 0 \\ 0 & 0.3 & 0 & 0.65 & 0 \\ 0 & 0 & 0.5 & 0 & 0 \\ 0 & 0 & 0 & 0.35 & 1 \end{matrix} \right] $$

I am trying to find the probability of the process starting at $s_2...s_4$ being absorbed by one of the absorbing states given infinite steps. Since $s_1$ and $s_5$ are absorbing states, after infinite steps the process will end up in either of the states so the sum of the two probabilities must equal one, but how can we find the distribution of the two states?

  • $\begingroup$ The same question was asked recently hence some other user is following the same course or studying the same book. What do you know about hitting probabilities? $\endgroup$ – Did Sep 30 '15 at 9:52
  • $\begingroup$ I think you refer to a question I asked yesterday. I deleted the question due to little response. I know very little about probability in general. This is for a project in linear algebra and I have not yet taken a statistics/probability course so I find it quite hard to follow. $\endgroup$ – C.T. Sep 30 '15 at 9:59
  • 5
    $\begingroup$ Do not re-ask the same questions. $\endgroup$ – Did Sep 30 '15 at 10:01
  • $\begingroup$ Your claim is wrong. It is not enough that there are absorbing states to be able to conclude that with probability one the process will end up in an absorbing state. You also must check that every state can reach some absorbing state with nonzero probability. (Also take a look at math.stackexchange.com/a/1293675/21820 for a rigorous proof of the correct claim.) $\endgroup$ – user21820 Sep 30 '15 at 12:08
  • $\begingroup$ Why are the states in columns instead of rows? Is this normal? $\endgroup$ – Roymunson Jan 16 at 18:48

Let $p_i$ be the probability that the process is eventually absorbed by $s_1$ after starting at $s_i$. Then $p_1=1$, $p_5=0$ and

\begin{align} p_2&=0.7p_1+0.3p_3\;,\\ p_3&=0.5p_2+0.5p_4\;,\\ p_4&=0.65p_3+0.35p_5\;. \end{align}

This system of three linear equations in three unknowns has a unique solution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.