# Calculating a stochastic matrix with multiple states

I am struggling with how to calculate the values of a Markov matrix which has multiple states.

For example,

Imagine an unfair 6 sided dice. The chance of rolling a 1,2,3,4,5 or 6 is

0.3, 0.25, 0.2, 0.12, 0.10, 0.03 respectively

The idea is that you keep rolling the dice until the accumulated total is greater than 6. That gives us 7 states, 1,2,3,4,5,6,7+

This gives me the following transition matrix

0.3, 0

0.25, ?

0.2, ?

0.12

0.10

0.03

0

The first column in the Markov matrix is simply the probabilities listed above along with 0 for the chance of rolling greater than a 6 on first throw. The next column obviously starts with 0 because it is not possible to still have an accumulated chance of 1 after 2 throws. After is where I get confused. I have no idea how to calculate the probability of being in state 2 after the second throw because this relies on current state being a 1 and then throwing another 1 (surely it isn't simply 0.3). Even more confusing to me is the probability of being in state 3. This means the current state either has to be 1 and roll a 2 or current state 2 and roll a 1.

If anyone could explain to me how to calculate the probability of state 2 in column 2 and state 3 in column 2 I would greatly appreciate it.

A Markov transition matrix contains the one-step transition probabilities from each state to every other state. With seven states your matrix is $7\times 7$, and if I have understood your process correctly, looks like:
$$\matrix{ &\color{red} 1&\color{red}2&\color{red}3&\color{red}4&\color{red}5&\color{red}6&\color{red}{7+}\cr \color{red}1& 0&.30&.25&.20&.12&.10&.03\cr \color{red}2& 0&0&.30&.25&.20&.12&.13\cr \color{red}3& 0&0&0&.30&.25&.20&.25\cr \color{red}4& 0&0&0&0&.30&.25&.45\cr \color{red}5& 0&0&0&0&0&.30&.70\cr \color{red}6& 0&0&0&0&0&0&1\cr \color{red}{7+}& 0&0&0&0&0&0&1\cr }$$