1
$\begingroup$

Given the probability matrix $P$ with states $s_1...s_5$ where $s_1$ and $s_5$ are absorbing

$$ P = \left[ \begin{matrix} 1 & 0.7 & 0 & 0 & 0 \\ 0 & 0 & 0.5 & 0 & 0 \\ 0 & 0.3 & 0 & 0.65 & 0 \\ 0 & 0 & 0.5 & 0 & 0 \\ 0 & 0 & 0 & 0.35 & 1 \end{matrix} \right] $$

I am trying to find the probability of the process starting at $s_2...s_4$ being absorbed by one of the absorbing states given infinite steps. Since $s_1$ and $s_5$ are absorbing states, after infinite steps the process will end up in either of the states so the sum of the two probabilities must equal one, but how can we find the distribution of the two states?

$\endgroup$
5
  • $\begingroup$ The same question was asked recently hence some other user is following the same course or studying the same book. What do you know about hitting probabilities? $\endgroup$
    – Did
    Sep 30, 2015 at 9:52
  • $\begingroup$ I think you refer to a question I asked yesterday. I deleted the question due to little response. I know very little about probability in general. This is for a project in linear algebra and I have not yet taken a statistics/probability course so I find it quite hard to follow. $\endgroup$
    – C.T.
    Sep 30, 2015 at 9:59
  • 5
    $\begingroup$ Do not re-ask the same questions. $\endgroup$
    – Did
    Sep 30, 2015 at 10:01
  • $\begingroup$ Your claim is wrong. It is not enough that there are absorbing states to be able to conclude that with probability one the process will end up in an absorbing state. You also must check that every state can reach some absorbing state with nonzero probability. (Also take a look at math.stackexchange.com/a/1293675/21820 for a rigorous proof of the correct claim.) $\endgroup$
    – user21820
    Sep 30, 2015 at 12:08
  • 1
    $\begingroup$ Why are the states in columns instead of rows? Is this normal? $\endgroup$
    – Foobar
    Jan 16, 2020 at 18:48

1 Answer 1

Reset to default
5
$\begingroup$

Let $p_i$ be the probability that the process is eventually absorbed by $s_1$ after starting at $s_i$. Then $p_1=1$, $p_5=0$ and

\begin{align} p_2&=0.7p_1+0.3p_3\;,\\ p_3&=0.5p_2+0.5p_4\;,\\ p_4&=0.65p_3+0.35p_5\;. \end{align}

This system of three linear equations in three unknowns has a unique solution.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .