# Questions tagged [transition-matrix]

A matrix associated to a transition of a Markov chain. The entries of this matrix represents a probability with the sum of a whole column being $1$.

130 questions
1answer
20 views

### simulating a discrete markov process from a reducible transition rate matrix

I'm trying to model an irreversible, discrete Markov process. I have a set of states $S$ arranged in a tree-like structure (it is only possible to move from parent vertex to child vertex). I compute ...
0answers
30 views

0answers
35 views

### estimate Markov chain mean transition time

Assume a continuous time Markov chain which is run through in one direction and finally absorbed at the last state $1 \rightarrow 2 \rightarrow 3 \rightarrow ... \rightarrow n$ The transition ...
0answers
15 views

### Same transition matrix implies markov chain.

Let $X_i$ , $i \geq 0$ be discrete time and state stochastic process. Suppose that $P(X_n = j | X_{n-1} = i)$ does not depend on $n$. that is $$P(X_n = j| X_{n-1} = i) = P(X_1 = j | X_0 = i)$$ for ...
0answers
34 views

### Transition probability matrix, weather changes

I am trying to solve the following problem: and the probability that there will be three sunny days in a row specifically. I know that the answer is $1/5$ but I am trying to figure out how to get ...
0answers
20 views

### Absorption probabilities of a Markov chain

So I was reading on Markov chains and came across this problem: $P$ is an $N\times N$ transition matrix such that $\sum_{j=0}^{N} jp_{ij}=i$ for all $1\leq i \leq N$. a) Prove that states 1 ...
0answers
39 views

0answers
29 views

### “Extra” step in counting the number of ways to have a domino tiling of a 4 by n rectangle?

The picture above explains a method for doing this via generating functions & finite state machines, but what I do not get is why we must record the number of dominoes used to make a state ...
0answers
14 views

### Using Time Homogenous Markov Chain to maximise profit

Suppose there is a company with 30 printing machines and they need to hire staff to operate the machines such that the profit is maximised. The model is that each machine is either 1. Idle state (not ...
1answer
19 views

### Computing eigenvectors from a transition matrix

I have a 21x21 transition matrix modeling the population of a species, and I'm trying to find the long term population proportions of the states. To do this, I'm using numpy. I found the dominant ...
1answer
350 views

### Transition Matrix from B to C

If $B=\{ 2+x,1+2x\}$ and $C=\{ 1+x, 1-x\}$ are 2 basis for $P_1$, and $v=-3x+4$ find $[v]_B$, $_BP_C$ and $[v]_C$. my attempt: Since $B$ is a basis for $V$, then any $v\in B$ can be written "...
0answers
40 views

1answer
199 views

### probability transition matrix markov chain

I got an exercise in matlab were i need to solve two things. I think i have solved the first part but struggle to solve part 2. Info: A small version of the game Snakes and Ladders is shown in the ...
2answers
83 views

### Find the transition matrix for a six-sided die

For some reason I have been struggling with this problem for the past couple hours. I believe I have solved part a. Since there are 6 states (assuming a standard die and the die is fair), then there ...
1answer
156 views

### Markov chain constant transition matrix

I am struggling with part B of this problem. I understand Markov chains and transition matrices but I'm stuck on where to start. Maybe it is just the wording of the problem. Can anybody point me in ...
0answers
52 views

### What is the difference between a transition function and a transition kernel?

Context of Question: "Algorithms for Reinforcement Learning", Csaba Szepesvari pg 11 Excerpt: In the case of the inventory control problem, the MDP was conveniently speci ed by a transition ...
1answer
100 views

### Change of Basis Given 2 Vectors and Transition Matrix

Given v$_1=\left( \begin{array}{ccc} 3\\ -4 \end{array} \right)$, v$_2=\left( \begin{array}{ccc} 2\\ 5 \end{array} \right)$, $S=\left( \begin{array}{ccc} -1 & 7\\ 2 &-5 \end{array} \right)$ ...
0answers
331 views

### Trying to grasp how to compute a $n$-step transition probability

As the title suggests, I am trying to understand how to compute a $n$-step transition probability given a transition probability matrix. Please understand that this is purely for me to prepare for ...