I am struggling with how to calculate the values of a Markov matrix which has multiple states.

For example,

Imagine an unfair 6 sided dice. The chance of rolling a 1,2,3,4,5 or 6 is

0.3, 0.25, 0.2, 0.12, 0.10, 0.03 respectively

The idea is that you keep rolling the dice until the accumulated total is greater than 6. That gives us 7 states, 1,2,3,4,5,6,7+

This gives me the following transition matrix

0.3, 0

0.25, ?

0.2, ?





The first column in the Markov matrix is simply the probabilities listed above along with 0 for the chance of rolling greater than a 6 on first throw. The next column obviously starts with 0 because it is not possible to still have an accumulated chance of 1 after 2 throws. After is where I get confused. I have no idea how to calculate the probability of being in state 2 after the second throw because this relies on current state being a 1 and then throwing another 1 (surely it isn't simply 0.3). Even more confusing to me is the probability of being in state 3. This means the current state either has to be 1 and roll a 2 or current state 2 and roll a 1.

If anyone could explain to me how to calculate the probability of state 2 in column 2 and state 3 in column 2 I would greatly appreciate it.

Thanks for your time.


A Markov transition matrix contains the one-step transition probabilities from each state to every other state. With seven states your matrix is $7\times 7$, and if I have understood your process correctly, looks like:

$$\matrix{ &\color{red} 1&\color{red}2&\color{red}3&\color{red}4&\color{red}5&\color{red}6&\color{red}{7+}\cr \color{red}1& 0&.30&.25&.20&.12&.10&.03\cr \color{red}2& 0&0&.30&.25&.20&.12&.13\cr \color{red}3& 0&0&0&.30&.25&.20&.25\cr \color{red}4& 0&0&0&0&.30&.25&.45\cr \color{red}5& 0&0&0&0&0&.30&.70\cr \color{red}6& 0&0&0&0&0&0&1\cr \color{red}{7+}& 0&0&0&0&0&0&1\cr }$$

Note: The red labels on the rows and columns are not matrix values.

  • $\begingroup$ Thank you very much. Can this matrix be used to calculate probabilities of being in a certain state depending on how many times you've thrown the dice? $\endgroup$ – codetemplar Jul 12 '15 at 17:44
  • $\begingroup$ Yes. You multiply the matrix by itself as many times as you roll the dice. $\endgroup$ – user940 Jul 12 '15 at 17:50
  • $\begingroup$ Tested it and got it working. Thank you so very much $\endgroup$ – codetemplar Jul 12 '15 at 19:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.