Skip to main content

Questions tagged [transition-matrix]

A matrix associated to a transition of a Markov chain. The entries of this matrix represents a probability with the sum of a whole column being $1$.

Filter by
Sorted by
Tagged with
0 votes
0 answers
17 views

Inverse Problems for Markov Models

Consider a Markov process with three states, whose transition scheme is represented as follows: The stated model includes four parameters, i.e., the transition rates $s_x$, $g_x$, $\mu^{N S}_x$, and $...
Pazu's user avatar
  • 1,077
0 votes
2 answers
22 views

Finding the coordinates related to the standard basis when given the coordinates relative to a different basis

The problem: Find the coordinates of the vector $A$ related to the standard basis for $M_2(\mathbb{R}) $ if it has the coordinates $(1, -1, 3, 2)$ relative to the basis $B = \left\{ E_1 = \begin{...
Darius's user avatar
  • 3
0 votes
0 answers
11 views

Continuous time Markov Processes, transition rate matrix Q

I have these lecture notes on continuous time Markov Processes: CTMPs I don't seem to understand the relation $Q=P'(0)$, for P: the transition probability matric and Q: the transition rate matrix. ...
martyna1222's user avatar
0 votes
0 answers
51 views

Calculate $P_{BB}^{n}$ using the periodicity of a Markov chain

I want to compute the probability of $P_{BB}^{n}$ with the following transition matrix: \begin{equation} \begin{pmatrix} 0 & 1/2 & 0 & 1/2 & 0 \\ 0 & 0 & 2/3 & 0 & 1/3 \...
Alex A.G.'s user avatar
  • 177
0 votes
0 answers
33 views

Transition matrix for defined Markov chain

Consider a sequence of iid random variables $ \lbrace \xi_n, \; n=0,1,2,\ldots\rbrace $ with mass probabilities $P(\xi_n=0)=0.1,P(\xi_n=1)=0.5, $ and $P(\xi_n=2)=0.4.$ Define a Markov chain $(X_n)_{n\...
waterr's user avatar
  • 71
0 votes
0 answers
47 views

Upper bound Frobenius norm of transition matrix

Suppose we have two transition probability matrices $\hat{\textbf{P}}$ and $\textbf{P}$. I'm searching for an upper bound on the Frobenius norm of their difference $$\|\hat{\textbf{P}} - \textbf{P}\|_{...
arlether's user avatar
0 votes
0 answers
29 views

Upper bound on largest eigenvalue of a sub-stochastic transition matrix

I would like to find an upper bound on the largest eigenvalue,$\lambda$, of a sub-stochastic transition matrix on set $S$, $P_{S}$, in terms of the stationary distribution of $P$, named $\pi$. Is it ...
Esa's user avatar
  • 1
4 votes
1 answer
40 views

Find the return time of a state using absorption probabilities in a finite Markov chain

Suppose we have a Markov chain with transition matrix $$\textbf{P}= \begin{pmatrix} \frac{1}{2} & \frac{1}{2} & 0&0&0&0 \\ \frac{1}{3} & \frac{2}{3} &0&0&0&0 \\...
Pranava Priyanshu's user avatar
0 votes
0 answers
48 views

Continuous time Markov chain

A continuous time Markov chain {$X(t); \ t \ \geq0$} has the state space $S_X = \ ${$0,1,2$} and the following transition matrix : $$\begin{bmatrix}-8 & 4 & 4 \\ 3 & -5 & 2 \\ 0 & ...
AANICR's user avatar
  • 93
0 votes
0 answers
20 views

Alternating Markov chains

Suppose I have a transition matrix (A) for a Markov chain. Now suppose there’s a second transition matrix (B) for another Markov chain over the same states. Now suppose I alternate between the two ...
Maximilian Nguyen's user avatar
0 votes
1 answer
140 views

How can I calculate the limit of a transition matrix?

I've been wondering about how can I calculate the limit of this matrix: These states the different movements of the knight within a $4\times3$ chessboard, and what I'm trying to do with the limit ...
Daniel Alonso Paz's user avatar
0 votes
0 answers
35 views

Prove all states are persistent

Suppose we have a time-homogeneous chain with a state space with $d$ states, where $d\in\mathbb N^+$. The transition matrix $P$ is given as below, which is a $d\times d$ matrix. $ P= \begin{bmatrix} ...
Sam Wong's user avatar
  • 2,287
2 votes
0 answers
67 views

Transition matrix between bases for polynomial vector spaces

Consider the bases $b=(p_1, p_2)$ and $b' = (q_1, q_2)$ for $p_1$, where $p_1 = 6 + 3x$, $p_2 = 10 + 2x$, $q_1 = 10 + 2x$, $q_1 = 2$, $q_2 = 3 + 2x$. Find the transition matrix from $B$ to $B'$. I've ...
Hedde Henessy's user avatar
1 vote
2 answers
150 views

Find the expected number of tosses to win a game

A friend and I play a game. We each start with two coins. We take it in turns to toss a coin; if it comes down heads, we keep it, if tails, we give it to the other. I always go first, and the game ...
drobin's user avatar
  • 21
0 votes
1 answer
54 views

Finding the probability of transitioning at exactly time n from a three-state transition matrix

First post here and trying to understand Markov chains/processes. Apologies if the notation used is non-standard but I hope it all makes sense. Let's assume we have a time homogeneous Markov process ...
Daniel Mhango's user avatar
0 votes
0 answers
44 views

Kullback--Leibler Divergence between Powers of Matrices

Consider a finite set $\mathcal{S}.$ Let $P$ and $Q$ be two transition probability matrices on $\mathcal{S} \times \mathcal{S}$, with each row summing to $1.$ Given $i,j \in \mathcal{S}$, let $P(i,j)$ ...
P. N. Karthik's user avatar
1 vote
1 answer
42 views

Markov chains and their distribution ruins

you chain. I'm having trouble wrapping my head around finding the stationary distribution when the state space is infinite. Anyone have any tips/advice or a solution? Thanks!
juggernaut's user avatar
1 vote
1 answer
310 views

Left and right eigenvectors of transition matrix relationship and normalization

I'm currently reading this article (1) and I want to verify equation (4) and (5). I will reproduce the claim here. Let $P$ be a transition matrix and $\phi_0$ be the stationary distribution of $P$. ...
Mevve's user avatar
  • 658
1 vote
1 answer
46 views

Competing predators modelled using continuous time Markov Chain.

Imagine we have 3 predators living in the same forest lets call them $P_1$, $P_2$ and $P_3$. They all live by themselves and see eachother as competitors, therefore if one predator meets another one ...
Max's user avatar
  • 157
1 vote
1 answer
287 views

General Two-State Continuous Markov Chain - Transition Probability Matrix not Valid.

We have the following transition rate matrix for a two-state Markov Chain: $$ Q = \begin{pmatrix} -\lambda_1 & \lambda_2 \\ \lambda_1 & -\lambda_2 \end{pmatrix} $$ Note I use the convention ...
Max's user avatar
  • 157
0 votes
1 answer
64 views

How can block-matrices be irreducible?

Suppose we define two matrices P1 and P2 as follows: both are 2x2 matrices both have strictly positive entries And then we define P to be a 4x4 transition matrix of the form P = [P1 0] [0 P2] It'...
mr. man's user avatar
  • 115
1 vote
1 answer
78 views

Rate of convergence of transition matrix when the probability of self edge is spread evenly across all other nodes.

Given a transition matrix $A$ of size $n$ (all elements are non-zero), if we construct another transition matrix $A'$ such that $A'[i][j] = A[i][j] + A[i][i]/(n-1) \,$ and $A'[i][i] = 0$ for all $0 \...
CuriousScientist's user avatar
0 votes
1 answer
34 views

Explanation of probabilities in discrete Markov chain process

Let say, we have a Marcov chain process (discrete) denoted by $\left[X_t\right], t=0,1,2,...$. This Marcov chain has 4 different ...
Bogaso's user avatar
  • 196
1 vote
0 answers
35 views

Evaluate $\mathbb{P}\left(X_n=1\,|\,X_0 = 1\right)$ where $\{X_n\}_{n\in \mathbb{N}_0}$ is a Markov Chain with a given transition matrix $P$.

Suppose we are given a time homogeneous Markov chain $\{X_n\}_{n\geq 0}$ with state space $\mathcal{X} = \{1,2,3\}$ and transition matrix $$P = \begin{pmatrix}0 & 1 & 0 \\ 0 & 2/3 & 1/...
ChrisNick92's user avatar
  • 2,932
2 votes
1 answer
189 views

Sum over rows and columns of double stochastic matrix

I have the following $2m \times 2m$ matrix $$\tilde{P}((u,v), (x,y)) = \begin{cases} \frac{1}{d_v -1}, & \text{if } v=x ~\text{and}~ y\neq u \\ 0 & \text{otherwise } \end{cases}$$ ...
Thoth's user avatar
  • 863
1 vote
0 answers
34 views

right-continuous at $0$ implies continuous for semi-group

I'm reading the book "Markov Chains : Gibbs Fields, Monte Carlo Simulation, and Queues" written by Pierre Brémaud In p.333 the author define transition semigroup as this : Let $(P_{t})_{t \...
ZENG's user avatar
  • 857
2 votes
1 answer
362 views

If all the elements in the main diagonal of a transition matrix are non-zero, is it possible that the Markov chain is not reversible?

I understand that a Markov chain is reversible if $\pi_{i}P_{ij} = \pi_{j}P_{ji}$ and I was looking for some examples of non-reversible Markov chains. I noticed that in all the examples I saw, at ...
Jack's user avatar
  • 55
1 vote
1 answer
120 views

Deriving transition probability matrix with reedefined states

I have a Markov chain with 4 states $\left(X_1, X_2, X_3, X_4\right)$ and the absorbing state as $\left(X_5\right)$ having the transition probability matrix as $ M= \begin{bmatrix} 0.60 & 0....
Brian19931's user avatar
1 vote
0 answers
33 views

show that $\mathbb{P}_\mu\left(X_{n+m}=y \mid X_m=x\right)=\mathbb{P}_x\left(X_n=y\right)=P^n(x, y)$

Let $P$ be a transition matrix on $E$, $\left(X_n\right)_{n \in \mathbb{N}}$ a Markov chain of transition matrix $P$, and $\mu$ a probability measure on $E$.The initial law of $X$ is $\mu$. Show that ...
vitalmath's user avatar
  • 285
0 votes
1 answer
108 views

How to calculate n-step (Markov chain) probability with excluding states?

Given the transition probability matrix $P=\begin{Vmatrix} 0.9 & 0.08 & 0.02\\ 0.85 & 0.10 & 0.05\\ 0 & 0 & 1 \end{Vmatrix},\qquad P^{(2)}=P^2=\begin{Vmatrix} 0.878 & 0.08 &...
Esther's user avatar
  • 33
0 votes
0 answers
33 views

Transition matrix of a single type branching process

Show that the branching process $\{X_n\}_{n\ge1}$ is a MC with state space $\mathbb{N}^+\cup\{0\}$. Find its transition matrix. The first part is easy and I could do it in two steps- Proved that $\...
reyna's user avatar
  • 2,224
0 votes
0 answers
244 views

Doubly stochastic matrix

Prove that Markov chain will be irreducible if its transition matrix is doubly stochastic. I was told to use Birkhoff's theorem, but I don't know how to use it at all.
Marja's user avatar
  • 37
0 votes
0 answers
168 views

For markov transition matrix and the initial state, calculate probability to reach certain other state in k or less steps

So there is a vector n giving the initial state and a Markov transition probability matrix M. I know I can calculate the ...
clime's user avatar
  • 101
1 vote
2 answers
67 views

Find the result of T given matrices against ordered basises

I am trying to solve this problem: Consider in $\mathbb{R}^2$ the ordered basis $v_1 = (1, 3)$, $v_2 = (2, 5)$. Let $T : \mathbb{R}^2 \to \mathbb{R}^2$ be a linear image with as the matrix with ...
fatCat9999's user avatar
0 votes
0 answers
26 views

value iteration update rule in MDP

Well, I'm new to MDP and I have a basic question about the formulation of the transition matrix (T). The way I would think about it when constructing T is an SxS matrix when S is the number of states. ...
Fady Guirguis's user avatar
1 vote
0 answers
27 views

Reconstruct transition matrix of an irreductible discrete-time markov process

Premise: I have searched the internet for the past 3 days without finding anything relevant. The problem I want to address is the following: I have an initial distribution D(0). I also have a certain ...
Kidara's user avatar
  • 11
0 votes
1 answer
87 views

Invariant distribution of a Markov chain - joint probability

I have found the following invariant distribution $\pi$, invariant w.r.t. my transition probability matrix: $$P = \begin{bmatrix} 0.8 & 0.2 \\ 0.5 & 0.5 \end{bmatrix}$$ $\pi = [0.7142857, 0....
yoyo_24's user avatar
  • 57
0 votes
0 answers
23 views

Calculating transition matrix for $P(X_{20}=2|X_{10}=2)$

Deduce the general form of $P^n$ in terms of $n$ and use this answer to calculate $P(X_{20}=2|X_{10}=2)$ $$P=\begin{pmatrix}\frac{1}{2}&\frac{1}{2}&0&0\\ \frac{1}{2}&\frac{1}{2}&0&...
tesla john's user avatar
0 votes
1 answer
85 views

For a better understanding of Markov Chains

I'm doing some exercises in probability on the markov chains, there is a task that I don't really understand the meaning of but I think it is very important to move on. It is as follows: Let $P$ be a ...
vitalmath's user avatar
  • 285
0 votes
1 answer
985 views

Power of Markov transition matrix

Given a Markov transition matrix $P$, what is the meaning of $P^n$? And what is the meaning of the $(i,j)$ entry of matrix $A=\sum_{k=0}^{k=n}{P^k}$? A possible answer is the expected number of times ...
Pap's user avatar
  • 35
0 votes
1 answer
1k views

Periodic and aperiodic states in a Markov chain

Imagine the following Markov chain: $$\begin{bmatrix} 0 & 0.5 & 0.5 \\ 1 & 0 & 0 \\ 1 & 0 & 0\end{bmatrix}$$ We always get back to state 1 in two time periods. So, state 1 is ...
Mostafa's user avatar
  • 148
1 vote
1 answer
69 views

Probability that a certain sequence is belonging to a certain transition matrix

I have the following transition matrices, one for Maria and one for Anna: ...
quant's user avatar
  • 111
1 vote
3 answers
870 views

Expected number of flips vs probability in a Markov Chain

The problem is the following: (a) We keep flipping coins until we see the sequence HTHH. Find the expected number of flips. (b) Alice and Bob play the following game. They keep flipping coins until ...
user13121312's user avatar
0 votes
0 answers
624 views

Find transition matrix for Markov Chain

The problem is the following: An individual has three umbrellas, some at her office, and some at home. If she is leaving home in the morning (or leaving work at night) and it is raining, she will ...
user13121312's user avatar
0 votes
1 answer
74 views

What is the meaning of the "transition" matrix associated to an order $2$ subshift dynamical system?

$\newcommand{\w}{\mathscr{W}}$I cite this text on dynamics, from chapter $2$ in the early pages, mainly section $2.4$. They use the convention that $\Bbb N$ does not include $0$. Some context: For $k\...
FShrike's user avatar
  • 41.2k
1 vote
1 answer
63 views

On the equivalence of different assumptions of Hidden Markov Models (HMM)

I am currently studying Hidden Markov Models (HMM). We denote the hidden quantities as $(X_0, \dots, X_n) \in \mathcal{X}^{n+1}$ and the observed quantities $(Y_1, \dots, Y_n) \in \mathcal{Y}^n$. The ...
rkvymvqt's user avatar
1 vote
0 answers
29 views

Reconstructing Markov chain from figure in R

I am trying to reconstruct a Markov process from Shannons paper "A mathematical theory of communication". My question concerns figure 3 on page 8 and a corresponding sequence (message) from ...
Steffen Schwerdtfeger's user avatar
1 vote
0 answers
151 views

Branching Process Understanding

I'm brand new to Markov chains, and a basic question I'm working on goes: Consider a branching process $\{X_n\}^\infty_{n=0}$ where $X_0=1, X_n=\sum^{X_{n-1}}_{i=1}Y_{i,n}$ for $n \ge1$, and $\{Y_{i,j}...
a9302c's user avatar
  • 349
2 votes
1 answer
335 views

Probability of getting absorbed at an odd-valued step

The problem that I'm working on is given below: A Markov chain $X_0,X_1,X_2,\ldots$ has the transition probability matrix: $$P=\begin{Vmatrix} 0.3 & 0.2 & 0.5\\ 0.5 & 0.1 & 0.4 \\ 0 &...
EzBots's user avatar
  • 303
0 votes
1 answer
394 views

Probability of a sum of two die rolls being 5 and not 7 in a game

I've been working on this problem for a while and I'm not sure how to approach it. This was given in a class on Stochastic Processes, and it's meant to be solved using a Markov chain: A single die is ...
EzBots's user avatar
  • 303

1
2 3 4 5 6