I am struggling with how to calculate the values of a Markov matrix which has multiple states.
Imagine an unfair 6 sided dice. The chance of rolling a 1,2,3,4,5 or 6 is
0.3, 0.25, 0.2, 0.12, 0.10, 0.03 respectively
The idea is that you keep rolling the dice until the accumulated total is greater than 6. That gives us 7 states, 1,2,3,4,5,6,7+
This gives me the following transition matrix
The first column in the Markov matrix is simply the probabilities listed above along with 0 for the chance of rolling greater than a 6 on first throw. The next column obviously starts with 0 because it is not possible to still have an accumulated chance of 1 after 2 throws. After is where I get confused. I have no idea how to calculate the probability of being in state 2 after the second throw because this relies on current state being a 1 and then throwing another 1 (surely it isn't simply 0.3). Even more confusing to me is the probability of being in state 3. This means the current state either has to be 1 and roll a 2 or current state 2 and roll a 1.
If anyone could explain to me how to calculate the probability of state 2 in column 2 and state 3 in column 2 I would greatly appreciate it.
Thanks for your time.