Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [transition-matrix]

A matrix associated to a transition of a Markov chain. The entries of this matrix represents a probability with the sum of a whole column being $1$.

1
vote
1answer
20 views

simulating a discrete markov process from a reducible transition rate matrix

I'm trying to model an irreversible, discrete Markov process. I have a set of states $S$ arranged in a tree-like structure (it is only possible to move from parent vertex to child vertex). I compute ...
0
votes
0answers
30 views

Can be a transition matrix of homogeneous Markov's Chain to be disjoint in t?

Can be a transition matrix of homogeneous Markov's Chain to be discontonuous of $t$, if $P(X_τ = j∣X_{t_n} = i_n, X_{t_n−1} = i_{n−1}, . . . , X_{t_1} = i_1) = P(X_τ = j∣X_{t_n} = i_n)$ $\forall t_1&...
0
votes
1answer
35 views

How do I compute the transition matrix P from two bases of 2x2 matrices?

I am given two bases of a vector space consisting of matrices. e basis: e1=`\begin{bmatrix}1&2\\0&5\end{bmatrix} e2=\begin{bmatrix}1&1\\-1&0\end{bmatrix} e3= \begin{bmatrix}1&0\...
0
votes
2answers
50 views

Probability of failure of a light bulb in years

Let's assume we have a lightbulb with a maximum lifespan 4 years. We are asked to create a transition matrix (Markov chain theory) for the bulb. The bulb is checked once a year and if it's found that ...
1
vote
0answers
13 views

Probabilites of different players winning a simplified 2-person version of ludo

Consider a game in which players A and B both start on square 0. Player A goes first, flipping a coin, and advances one square if heads (stays on same square if tails). Then Player B goes does ...
0
votes
1answer
18 views

Is there any concrete connection between a regular transition matrix and aperiodicty and irreducibility of a finite-state Markov Process?

The transition matrix T = \begin{bmatrix} 3/4 &1/4 \\ 1 &0 \end{bmatrix} is clearly a regular transition matrix but the chain itself is not aperiodic (although it is irreducible), right? (...
-1
votes
1answer
74 views

What would be the transition for ten coins?

There are ten coins and a move is made up of flipping any three adjacent coins: H H T T H T H T H T -> H T H H H T H T H T (flip: 2,3,4). How can this transition be represented? / EG: There are 102 ...
0
votes
0answers
12 views

Transition density of hidden Markov model

Background info: We have a target moving in $\mathcal R^2$ according to the model; $$X_{n+1}=\Phi X_n+\Psi_z Z_{n}+\Psi_w W_{n+1}, \quad n \in \mathcal{N}$$ The information contained in $X_n=\{X_n^1,...
0
votes
0answers
35 views

estimate Markov chain mean transition time

Assume a continuous time Markov chain which is run through in one direction and finally absorbed at the last state $1 \rightarrow 2 \rightarrow 3 \rightarrow ... \rightarrow n $ The transition ...
0
votes
0answers
15 views

Same transition matrix implies markov chain.

Let $X_i$ , $i \geq 0 $ be discrete time and state stochastic process. Suppose that $P(X_n = j | X_{n-1} = i) $ does not depend on $n$. that is $$P(X_n = j| X_{n-1} = i) = P(X_1 = j | X_0 = i) $$ for ...
1
vote
0answers
34 views

Transition probability matrix, weather changes

I am trying to solve the following problem: and the probability that there will be three sunny days in a row specifically. I know that the answer is $1/5$ but I am trying to figure out how to get ...
0
votes
0answers
20 views

Absorption probabilities of a Markov chain

So I was reading on Markov chains and came across this problem: $P$ is an $N\times N$ transition matrix such that $\sum_{j=0}^{N} jp_{ij}=i$ for all $1\leq i \leq N$. a) Prove that states 1 ...
1
vote
0answers
39 views

If $A$ and $B$ are transition matrices such that $||A-B|| < c$, then what can we say about $||A^n-B^n||$ for a given $n$?

Suppose there are two matrices, $A$ and $B$, that are both transition matrices for a Markov chain ($n\times n$, non-negative and row-stochastic). I know that A and B are "close" in the sense that $||A-...
0
votes
1answer
33 views

Long run proportion of transitions in a Markov chain

Let $S$ be a set of states for a Markov chain and let $S^C$ be the remaining states. Explain the identity $$\sum_{i\in S}\sum_{j\in S^C}\pi_iP_{ij}=\sum_{i\in S^C}\sum_{j\in S}\pi_iP_{ij}$$ I know ...
1
vote
1answer
44 views

Finding the transition probability matrix

$$P(ξ_i = k) = 1/m$$ for $$k = 1, 2, . . . , m.$$ Explain why $(X_n)_{n≥0}$is a Markov chain. Write down the state space and the transition probability matrix of $(X_n)_{n≥0}$.
0
votes
0answers
11 views

How to obtain Z score matrix from transition probability matrix?

I have a transition matrix as follows: A B C D A 99.7 0 0 0.3 B 0 68.6 20 11.4 C 0 47.2 23 29.8 D 0 20 15 65 I have come ...
1
vote
0answers
38 views

Markov process intensity matrix

$X$ is a Markov process with state space $(1,2,3)$. How can I find the matrices of transition probabilities $P(t)$ if the generator is \begin{bmatrix}-2&2&0\\2&-4&2\\0&2&-2\end{...
0
votes
1answer
17 views

If a transition matrix $A$ is regular, prove that $A^∞$ has all the same columns and that the columns are the steady state vector.

I know that this is quite an elementary theorem, but I have yet to see a proof of this except for quoting that $A^∞$ is going to be of rank 1, so that all the columns must be the same. Any help is ...
0
votes
1answer
25 views

Markov chain. Is steady state a scaled eigenvector of transition probability matrix

So suppose we have transition matrix P for a Markov chain and suppose it satisfies the relevant criteria so that $$ \lim_{n\rightarrow \infty} P^{(n)} = \pi $$ is well behaved and is some steady ...
-1
votes
1answer
41 views

Markov chain duration [closed]

What is the formula to find average duration of state s in a Markov chain given a transition matrix? I tried to recall the concept but could not find any references.
0
votes
1answer
19 views

Image of a Polynomial Basis

Let $B$ $=$ {$x^2, x, 1$} and $S$ $=$ {$x^2+x, 2x-1, x+1$} be two basis of $P_2$. Let $T$ be a linear transformation from $P_2$ to $P_2$ such that the transition matrix from $B$ to $S$ is $\begin{...
0
votes
0answers
29 views

“Extra” step in counting the number of ways to have a domino tiling of a 4 by n rectangle?

The picture above explains a method for doing this via generating functions & finite state machines, but what I do not get is why we must record the number of dominoes used to make a state ...
0
votes
0answers
14 views

Using Time Homogenous Markov Chain to maximise profit

Suppose there is a company with 30 printing machines and they need to hire staff to operate the machines such that the profit is maximised. The model is that each machine is either 1. Idle state (not ...
0
votes
1answer
19 views

Computing eigenvectors from a transition matrix

I have a 21x21 transition matrix modeling the population of a species, and I'm trying to find the long term population proportions of the states. To do this, I'm using numpy. I found the dominant ...
3
votes
1answer
350 views

Transition Matrix from B to C

If $B=\{ 2+x,1+2x\}$ and $C=\{ 1+x, 1-x\}$ are 2 basis for $P_1$, and $v=-3x+4$ find $[v]_B$, $_BP_C$ and $[v]_C$. my attempt: Since $B$ is a basis for $V$, then any $v\in B$ can be written "...
0
votes
0answers
40 views

How to calculate transition probability for Markov chain?

user mode diagram There is total $N$ users. For Markov chain, I set the number of $B$ mode users as state $x(t)$ at $t$. Each user is in a mode $A, A', B, \text{or } B'$. So, sum of users with $A, A', ...
0
votes
0answers
38 views

Transition probability of markov chain

I want to find transition probabilities of a Markov chain. However, one step probability of state transition is not a fixed but is dependent on the number of users with specific state. Below is detail:...
0
votes
1answer
58 views

How to calculate cost in discrete Markov transitions

I am not sure how to handle this problem where there are two types of cost: ( a ) cost of remaining in a state ( state_from = state_to ) ( b ) cost of transitioning from one state to another (...
0
votes
0answers
16 views

Transistion Matrix size with Coordinate

Context: I was given two 4x4 Matrices, A and B and I found their respective Basis, S and T, where the S the basis of A has 3 vectors while the T the basis of B has 2 vectors. I was then given a vector ...
0
votes
0answers
190 views

Distributions over probability simplex

I was wondering if there any probability distributions over the probability simplex ($p \in \mathbb{R^n_+}:\sum_i^n p_i=1,p_i\geq 0$). In particular, what are the distributions which can be used to ...
0
votes
1answer
87 views

Markov chain with three states and equal fractions

This is a homework problem and I have been stuck at it for over an hour. Any hint will be appreciated. The question states that a town is running a bike sharing program. A bike could be grabbed at a ...
0
votes
1answer
65 views

How does state transition matrix indicate time-varying system, but $A$ matrix is constant?

When I determine whether a linear dynamical system is time-varying or not by looking at the state transition matrix, I get confused by the following: Normally, I look at the $A$ matrix to determine ...
2
votes
1answer
52 views

How can I indicate that empty elements of a matrix are zero?

I have the following transition matrix in an academic conference paper: $$\mathbf{P}=\begin{bmatrix} 1-p & p & & & & \\ & 1-p & p &...
0
votes
0answers
35 views

Transition matrices with 2 bases

My question is about Transition matrices, I want to ask the following question: Given a Linear transformation $T:R^2\to R^2,$ Such that: $$T(x, y) = (3x-y, x-2y).$$ Let $B$ be basis for $R^2$ such ...
0
votes
0answers
161 views

Ergodic Markov chains and eigenvalues

I just read on wikipedia that a way to check whether a Markov chain is ergodic is to compute the eigenvalues of the transition matrix, and if those are all (except for 1) less than 1, then the chain ...
0
votes
1answer
32 views

How to derive the probability that a first return occurs at the $n$-th step from the transition diagram.

For a four state $(s_1, s_2, s_3, s_4)$ Markov chain, the transition probability matrix is given by: $$P = \begin{bmatrix} 1-a & a & 0 & 0\\ 1-b & 0 & b & 0\\ 1-c & 0 &...
2
votes
0answers
413 views

Proof of the existence of a unique stationary distribution in a finite irreducible Markov chain.

I am currently trying to understand a proof for the above, which states that, in other words, there exists a unique $\overrightarrow{v}$, such that $\overrightarrow{v}P = \overrightarrow{v}$ for the ...
1
vote
4answers
39 views

Finding $n$-th power of transition matrix

Is there any shortcut to find $P^{n}=\begin{pmatrix} 1-p & p \\ q & 1-q \end{pmatrix}^n$ quickly and elegantly? This type of matrix often comes up while dealing with Markov chains. ...
0
votes
1answer
32 views

fundamental matrix solution for difference equation

I need your help please, we have the system X(t+1)= \begin{bmatrix} -1 & \frac{2+(-1)^{t}}{2} \\ \frac{2+(-1)^{t}}{2} & -1 \end{bmatrix} X(t) By using this formula $\phi(t)=A(t-1)...A(1)A(0)$...
2
votes
1answer
109 views

Transition probability and probability for first visit

Let P be a transition matrix of a Markov Chain. Let $p_{ij}^{n}=P\left [ X_{n}=j | x_{0}=i\right ]$ be the transition probability of a Markov Chain from an initial state i to a final state j in n-...
0
votes
0answers
33 views

Matrix exponential of an LTI system

Is there any direct proof to show that if eigenvalues of an LTI system are negative then the transition matrix ( or matrix exponential with respect to time) e^{At} decays to zero when t goes to ...
2
votes
1answer
175 views

Is the given sequence a Markov chain or not? If yes, find its transition probabilities.

Let $\xi_n$, $n \in \mathbb{Z}_+$ be a sequence of i.i.d random variables over $\mathbb{R}$ with the density $p(x)$. Consider the sequence $$ \eta_n := \sum\limits_{k=1}^n\left(a\xi_k + b\xi_{k + 2}\...
1
vote
0answers
23 views

Can an autoregressive process of order $k$ be expressed as a $k$-step Markov chain?

I am curious if an autoregressive process of order $k:$ $X_{t}= c+ \sum_{i=1}^{k}\phi_i X_{t-i} + \epsilon_i$ can be expressed as a $k$-step Markov chain with transition probability $$ P_{ij}^{k} = ...
1
vote
1answer
76 views

Understanding proof of positivity of invariant distribution

$\mathbf{Theorem}$: Suppose $X$ is an irreducible Markov chain with transition matrix $P$. Let $\lambda$ be an invariant measure for $P$, i.e. $\lambda P = \lambda$. Suppose that some $\lambda_k > ...
2
votes
1answer
199 views

probability transition matrix markov chain

I got an exercise in matlab were i need to solve two things. I think i have solved the first part but struggle to solve part 2. Info: A small version of the game Snakes and Ladders is shown in the ...
2
votes
2answers
83 views

Find the transition matrix for a six-sided die

For some reason I have been struggling with this problem for the past couple hours. I believe I have solved part a. Since there are 6 states (assuming a standard die and the die is fair), then there ...
0
votes
1answer
156 views

Markov chain constant transition matrix

I am struggling with part B of this problem. I understand Markov chains and transition matrices but I'm stuck on where to start. Maybe it is just the wording of the problem. Can anybody point me in ...
1
vote
0answers
52 views

What is the difference between a transition function and a transition kernel?

Context of Question: "Algorithms for Reinforcement Learning", Csaba Szepesvari pg 11 Excerpt: In the case of the inventory control problem, the MDP was conveniently speci ed by a transition ...
1
vote
1answer
100 views

Change of Basis Given 2 Vectors and Transition Matrix

Given v$_1=\left( \begin{array}{ccc} 3\\ -4 \end{array} \right)$, v$_2=\left( \begin{array}{ccc} 2\\ 5 \end{array} \right)$, $S=\left( \begin{array}{ccc} -1 & 7\\ 2 &-5 \end{array} \right)$ ...
0
votes
0answers
331 views

Trying to grasp how to compute a $n$-step transition probability

As the title suggests, I am trying to understand how to compute a $n$-step transition probability given a transition probability matrix. Please understand that this is purely for me to prepare for ...