Questions tagged [transition-matrix]

A matrix associated to a transition of a Markov chain. The entries of this matrix represents a probability with the sum of a whole column being $1$.

Filter by
Sorted by
Tagged with
1
vote
1answer
63 views

Diagonalisation of stochastic matrices

Suppose that $(X_n)_{n≥0}$ is a Markov chain on a state space $I = {1, 2}$ and stochastic matrix $$P = \begin{bmatrix} \frac{1}{4} & \frac{3}{4} \\ \frac{1}{3} & \frac{2}{3} \end{bmatrix}$$ (a)...
3
votes
1answer
40 views

Are finite state irreducible continuous Markov chains identifiable in general?

Let $S=\{1,...,h\}$ be a finite state space and $X(t)$ an irreducible Markov chain fully described by a generator matrix $Q$ with a transition probability matrix $P(t)=e^{Qt}$ on time horizon $[0,T]$. ...
1
vote
1answer
55 views

Transition matrix exercise

I found this exercise on the internet ( I translated it from French so sorry if it's scuffed. ) I have no idea how to start it, any hint would be appreciated. Let $(X_n)$ be a Markov chain with $Q$ ...
0
votes
1answer
44 views

What is the expected profit?

I have been working on the following problem: The transition matrix of a driver moving in zone 1 to zone 2 and between the zones is given by P= [0.8 0.2, 0.3 0.7]. The state space is {zone1, zone2} ...
0
votes
0answers
28 views

Set up the transition probability matrix for flip a quarter until the pattern $HHT$ appears

You are going to successively flip a quarter until the pattern $HHT$ appears; that is, until you observe two successive heads followed by a tails. In order to calculate some properties of this game, ...
0
votes
0answers
34 views

Construct transition probability matrix: repeat toss a coin until two head or two tails consecutively.

Given an experiment: repeat toss a coin until two head or two tails consecutively. Example: HTT, THH, HTHH, THTHTT, etc. Construct transition probability matrix. I spent many time to think this ...
0
votes
1answer
30 views

How to find coordinates of vectors [x]e and [y]u in another basis

If we have basis {e1=(2,-1,-1),e2=(3,1,1),e3=(-2,-1,-2)} and basis {u1=(-3,1,2),u2=(1,1,3),u3=(-2,-2,-1)} The question is prove that e1,e2,e3 and u1,u2,u3 forms basis of R³ And find the transition ...
1
vote
0answers
45 views

Investigate the dice game using Markov chains. What is the probability of winning? What is the expected number of rolls?

Consider the following game: You start with a score of zero. We set a goal score of M. On each turn, you roll a six-sided, fair die. If your score is greater than zero and your roll divides your score,...
2
votes
0answers
204 views

Specify the classes of the following Markov chains, and determine whether they are transient or recurrent

Specify the classes of the following Markov chains, and determine whether they are transient or recurrent: $$\mathbb{P}_1=\begin{Vmatrix}0 & 1/2 & 1/2\\ 1/2 & 0 & 1/2\\ 1/2 & 1/2 &...
0
votes
1answer
21 views

Intuition on transition rate matrix of Continuous Time Markov Chain

According to wiki this is the definition: def of transition rate matrix That I do not get is why each row needs to sum up to 0, and why $q_{i,i}$ is negative $\lambda_i.$ Thank you
0
votes
0answers
19 views

Forward Transition Matrix Inhomogeneous MC

Given an inhomogeneous MC where the generator is ie a piece-wise constant function of t. I wonder under which condition is possible to get a valid forward transition matrix, given the cumulated. Given ...
0
votes
1answer
22 views

Markov chain generator vs. transition matrix

I have seen (in e.g. this paper) discussion of "Markov generators" $Q$ which are distinct from the transition matrices $P$ which I am more familiar with. One crucial difference is that $Q$ ...
1
vote
1answer
54 views

How can I write this Markov chain problem in a transition probability matrix form?

Mister X has a memory problem.Every night he forgets some people that he knows. More specifically, if he remembers $i$ people before he goes to sleep, the next morning he might remember $0,1,2,\dots,i$...
0
votes
0answers
7 views

Question about computing the probability of a transition $r \rightarrow r$ in a continous Markov process

I am using an R package called MSM to fit a continous time Markov model to a dataset describing transitions between a finite number of states. I am interested in ...
0
votes
0answers
40 views

Find the transition matrix of a coalescent random walk

Background Let $\mathcal{G} = (V, E)$, be an undirected and connected graph, with $N$ vertices. In a coalescing random walk, a set of particles make independent discrete-time random walks on a graph. ...
1
vote
0answers
64 views

Finding the general stationary distribution of a Markov chain

Consider a Markov chain with state space $$S = \{1, 2, 3, 4, 5\}$$ and transition matrix $$P = \begin{pmatrix} 0 & \frac 1 4 & \frac 1 4 & \frac 1 4 & \frac 1 4\\ \frac 1 2 & 0 &...
1
vote
2answers
65 views

Chapter 2 Exercise 2 Question (a) Page 84 Linda J. S. Allen 2010

Exercise 2 Question (a) Page 84 Textbook: An Introduction to Stochastic Processes with Applications to Biology 2nd Edition Linda J. S. Allen 2010 Exercise Suppose $P$ is an $N\times N$ stochastic ...
2
votes
1answer
59 views

Chapter 2 Exercise 1 Pages 83 Linda J. S. Allen 2010

Exercises for Chapter 2 Exercise 1 Pages 83 textbook: An Introduction to Stochastic Processes with Applications to Biology 2nd Edition Linda J. S. Allen 2010 Link to the textbook My attempt: $P=(p_{...
1
vote
1answer
40 views

Constructing a markov chain and giving its transition probabilities

I am having trouble understanding the question and I am not sure if I am heading in the right direction. This is how I have the transition probability matrix: $$s+1<j<S$$ \begin{array}{c|c|c|c} ...
1
vote
1answer
99 views

Question on transition probability matrices

Question: $P$ is the transition matrix of a finite state space Markov chain. Which of the following statements are necessarily true? $1.$ If $P$ is irreducible, then $P^2$ is irreducible. $2.$ If $P$ ...
0
votes
1answer
109 views

Question on Markov-Chain GATE (ST)-$2021$

Question: Let $\{X_n:n \ge0 \}$ be a time- homogeneous discrete time Markov-chain with either finite or countable state space $S$. Then $1.$ there is at least one recurrent state $2.$ if there is an ...
0
votes
1answer
94 views

A problem on Markov-chain CSIR-NET(JRF) Dec-$2011$

Question: Let $P$ be the stationary transition probability matrix of the Markov Chain $\{X_n:n\ge0\}$ which is irreducible and every state has period $2$. Further, suppose that Markov Chain $\{Y_n:n\...
0
votes
0answers
13 views

Jacobian columns make basis for tangent spaces. Is there a relation between the signs of the basis transition matrix and of the transition jacobian?

Let M be a smooth manifold. For $p\in M$, assume f and g are local coordinate systems for M aroud p. For some x,y we have $f(x)=g(y)=p$. The columns of $J_{f}(x)$ and the columns of $J_{g}(y)$ provide ...
2
votes
0answers
39 views

Balls in an urn - Transition matrix

There are 2 colorless balls in an urn. In a sequence of events, a ball is randomly chosen and painted in red or black, then we put it back in the urn. We do it again, if the ball is painted, we put ...
0
votes
1answer
38 views

Finding transition probabilities of a Markov chain exiting a specific state if it never enters that state

Title may be a bit confusing, allow me to explain: Say our state space $S$ = $\{1, 2, 3, 4\}$ Now let us say that we observe a Markov Chain for 6 periods, and find that its path is $\{1, 3, 4, 1, 3, 2\...
1
vote
0answers
91 views

Markov chain: probability of getting to one of the states before another

Let's say we have a 3-state CTMC represented by $$ \require{enclose} \begin{array}{ccc} & & \Large{\enclose{circle}{1}} \\\ & \lambda_1 \Large{\nearrow} & \\\ \Large{\enclose{...
0
votes
1answer
138 views

Markov process with Weibull transition rate

A discreet-time Markov process for which a transition probability matrix $P$ is independent of time can be represented, or approximated, with a continuous-time Markov chain (CMTC) with constant ...
0
votes
1answer
66 views

Prove that $U$ and $V$ are two bases of $\mathbb{R}^{3},$ find the transition matrice $P_{U\rightarrow V}$ and $P_{V\rightarrow U}$

In the vector space $\mathbb{R}^{3},$ given two systems of vectors $$U= \left \{ u_{1}= \left ( 4, 2, 5 \right ), u_{2}= \left ( 2, 1, 3 \right ), u_{3}= \left ( 3, 1, 3 \right ) \right \}$$ $$V= \...
0
votes
1answer
45 views

How do we calculate $P(X_5 = 2 \mid X_3 = 0, X_1 = 1)$?

I have the following problem: Bob is a salesman. Each week, Bob will either make no money, make a small amount of money, or make a large amount of money. And if Bob makes money, he will either make \$...
0
votes
1answer
27 views

What does it mean for one to calculate an approximate value for $\mathcal{P}(t)$ for a given value of $t$, and how is it done?

Let's say we have some Markov process $(X(t): t \ge 0)$ with state space $S = \{ 1, 2, 3 \}$. Furthermore, let's say we're also given a generator matrix $$Q = \begin{bmatrix} -1 & 0 & 1 \\ 4 &...
2
votes
0answers
153 views

What would be the state transition matrix $\Phi(t,0)$ for the linear system $\dot{x}(t) = A(t)x(t) + Bu(t)$ with a periodic $A(t)$.

I am trying to compute the state transition matrix $\Phi(t,0)$ for the following linear time-varying system of the form $$\dot{x}(t) = A(t)x(t) + Bu(t)$$ with a periodicity in $A(t)$. The state ...
0
votes
0answers
18 views

Regularity of the fixed point of a Kernel

Take $S\subset \mathbb R^n$ bounded and be $k(\cdot|s)$ a probailiplity density kernel, i.e. $$\int_S k(s'|s)\ ds'=1\qquad s-a.e.$$ What are the hypothesis I have to put on $k$ so that, for any $b(s)\...
0
votes
0answers
29 views

Initial Probability Distribution of a Markov Chain

c. If the initial probability distribution is $Pr [𝑋_0 = 𝑖] = 1/ 3; i= 1,2,3.$ Find the probability distribution of $𝑋_1.$ d. Suppose the process begins at in state $𝑋_0 = 1.$ Find the probability ...
0
votes
0answers
20 views

Proof that $\boldsymbol{P}_{i j}=\boldsymbol{P}_{j j}$ t for an irreducible DTMC with a countably infinite number of states and idempotent TPM 𝑷.

Suppose we have an irreducible DTMC with a countably infinite number of states and an idempotent Transition probability matrix (TPM) P. I want to prove that for all $\boldsymbol{i}$ and $\boldsymbol{j}...
4
votes
3answers
344 views

How can a markov transition matrix have eigenvalues other than 1?

A Markov transition matrix has all nonnegative entries and so by the Perron-Frobenius theorem has real, positive eigenvalues. In particular the largest eigenvalue is 1 by property 11 here. Furthermore ...
0
votes
1answer
151 views

The probability a Markov Chain never reaches a state

Given a Discrete Time Markov Chain and an initial distribution, how do you find the probability the chain will never reach a state? For example, an easy DTMC, knowing that it started at state 0, what ...
0
votes
0answers
22 views

Transition Matrix Acting on an Eigen Function

For background, I am reading a proof of a Theorem which states: For a reversible, irreducible, and aperiodic Markov chain, \begin{equation*} t_{\text{mix}}(\epsilon) \geq (t_{\text{rel}}-1)\log\biggr(\...
0
votes
2answers
52 views

Finding a transition matrix from basis B1 to the standard basis S.

This is the info I have: B1 = {(3,4,0), (-2,-1,1), (1,0,3)} S is the standard basis of $R^3$ Is the matrix supposed to look like this then? $$\begin{bmatrix}1 & 0 & 0 & 3 & 4 & 0\...
1
vote
0answers
42 views

Meaning of a matrix equation modelling migration probabilities

I'm a biologist and I'm working on a mathematical model that describe similarity between colonies (i.e. cells) occupying a circular habitat. Migration occur only between adjacent colonies at rate <...
2
votes
0answers
33 views

Cover time in Markov Chain from transition matrix

Given a process on a graph $X_{n} = \{x_{1}, ..., x_{n}\}$, is there a way to obtain the cover time, starting at any state $x_{i}$, from the transition matrix $\mathbf{P}$? I've obtained the expected ...
8
votes
2answers
348 views

Distribution of $ad-bc$

I'm interested in the stochastic process $$f_t=(ad-bc)_t$$ where $(a,b,c,d)_t$ is governed by the following transition rules: $$\begin{align}(a,b,c,d) \rightarrow \begin{cases} (a+1,b,c,d) \;\;\;\; \...
2
votes
2answers
89 views

A 4-dimensional Markov chain

Consider a 4-dimensional Markov chain with states $(a,b,c,d) \in \mathbb{N}^4$. The transition rule is the following: \begin{align}(a,b,c,d) \rightarrow \begin{cases} (a+1,b,c,d) \;\;\;\; \text{ with ...
0
votes
0answers
156 views

Finding transition probabilities from an intensity/transition rate matrix for Markov Chains.

I've just started studying Markov chains this year using my brother's old notes from college and I've been finding them quite useful so far. However, there is one thing that I'm not that sure of. If I'...
2
votes
0answers
33 views

Interpretation of the Sinkhorn-Knopp algorithm applied to a (singly stochastic) transition matrix of a Markov process?

Say I have a discrete-time Markov process (and let's say discrete states too, for simplicity). If $\mathbf p_t$ is a vector of probabilities over states at time $t$, then the probability distribution ...
0
votes
0answers
15 views

How to determine whether a transition count matrix corresponds to an actual sequence of labels?

If we observe weather conditions for 101 days and mark each day with one out of three labels (Sunshine, Rain, Snow, e.g.: … — Sunshine — Sunshine — Sunshine — Rain — Sunshine — Snow — Sunshine — Rain —...
1
vote
2answers
76 views

What does it mean for a stationary distribution to have imaginary values?

Let's say I have a transition matrix P $$ \begin{bmatrix} 0.4 & 0.6 & 0. & 0.& 0.\\ 0.5 &0. & 0.5 &0. & 0. \\ 0.6 &0. & 0. & 0.4 &0. \\ 0.7 &0. &...
1
vote
0answers
62 views

Asymptotic number of certain transition in a two-state Markov chain

Having a two-state Markov chain with the symmetric transition matrix: \begin{pmatrix} 1-p&p\\ p&1-p \end{pmatrix} The states are 1 and 2, let $n_{2\rightarrow1}(t)$ be the number of ...
0
votes
0answers
41 views

Stationary distribution of an ergodic Markov chain

Let $S_i$ be a Markov chain with transition Matrix $ P $ $ (0 ≤ p ≤ 1) $ \begin{equation*} P = \begin{pmatrix} 1-2p & 2 p& 0 \\ p & 1-2p & p \\ 0 & 2p & 1-2p \end{pmatrix} \...
2
votes
0answers
21 views

Compute backward transition density from forward density

For a continuous (forward) transition density $p(x_{t+1}|x_t)$, is there a general rule how to compute the backward transition density $p(x_t|x_{t+1})$ ? I am working on a discrete time grid $t=1,2,......
2
votes
0answers
30 views

Minimal exponent for transition matrix to be positive

For an irreducible Markov Chain on a finite state space we have that it is aperiodic if and only if there is an integer $k$ such that the transition matrix satisfies $P^k>0$, i.e., all entries of $...

1
2 3 4 5