Questions tagged [transition-matrix]

A matrix associated to a transition of a Markov chain. The entries of this matrix represents a probability with the sum of a whole column being $1$.

238 questions
Filter by
Sorted by
Tagged with
63 views

Diagonalisation of stochastic matrices

Suppose that $(X_n)_{n≥0}$ is a Markov chain on a state space $I = {1, 2}$ and stochastic matrix $$P = \begin{bmatrix} \frac{1}{4} & \frac{3}{4} \\ \frac{1}{3} & \frac{2}{3} \end{bmatrix}$$ (a)...
40 views

Are finite state irreducible continuous Markov chains identifiable in general?

Let $S=\{1,...,h\}$ be a finite state space and $X(t)$ an irreducible Markov chain fully described by a generator matrix $Q$ with a transition probability matrix $P(t)=e^{Qt}$ on time horizon $[0,T]$. ...
55 views

Transition matrix exercise

I found this exercise on the internet ( I translated it from French so sorry if it's scuffed. ) I have no idea how to start it, any hint would be appreciated. Let $(X_n)$ be a Markov chain with $Q$ ...
44 views

What is the expected profit?

I have been working on the following problem: The transition matrix of a driver moving in zone 1 to zone 2 and between the zones is given by P= [0.8 0.2, 0.3 0.7]. The state space is {zone1, zone2} ...
28 views

Set up the transition probability matrix for flip a quarter until the pattern $HHT$ appears

You are going to successively flip a quarter until the pattern $HHT$ appears; that is, until you observe two successive heads followed by a tails. In order to calculate some properties of this game, ...
34 views

Construct transition probability matrix: repeat toss a coin until two head or two tails consecutively.

Given an experiment: repeat toss a coin until two head or two tails consecutively. Example: HTT, THH, HTHH, THTHTT, etc. Construct transition probability matrix. I spent many time to think this ...
30 views

How to find coordinates of vectors [x]e and [y]u in another basis

If we have basis {e1=(2,-1,-1),e2=(3,1,1),e3=(-2,-1,-2)} and basis {u1=(-3,1,2),u2=(1,1,3),u3=(-2,-2,-1)} The question is prove that e1,e2,e3 and u1,u2,u3 forms basis of R³ And find the transition ...
45 views

Investigate the dice game using Markov chains. What is the probability of winning? What is the expected number of rolls?

Consider the following game: You start with a score of zero. We set a goal score of M. On each turn, you roll a six-sided, fair die. If your score is greater than zero and your roll divides your score,...
204 views

65 views

Chapter 2 Exercise 2 Question (a) Page 84 Linda J. S. Allen 2010

Exercise 2 Question (a) Page 84 Textbook: An Introduction to Stochastic Processes with Applications to Biology 2nd Edition Linda J. S. Allen 2010 Exercise Suppose $P$ is an $N\times N$ stochastic ...
59 views

13 views

Jacobian columns make basis for tangent spaces. Is there a relation between the signs of the basis transition matrix and of the transition jacobian?

Let M be a smooth manifold. For $p\in M$, assume f and g are local coordinate systems for M aroud p. For some x,y we have $f(x)=g(y)=p$. The columns of $J_{f}(x)$ and the columns of $J_{g}(y)$ provide ...
39 views

Balls in an urn - Transition matrix

There are 2 colorless balls in an urn. In a sequence of events, a ball is randomly chosen and painted in red or black, then we put it back in the urn. We do it again, if the ball is painted, we put ...
38 views

27 views

42 views

Meaning of a matrix equation modelling migration probabilities

I'm a biologist and I'm working on a mathematical model that describe similarity between colonies (i.e. cells) occupying a circular habitat. Migration occur only between adjacent colonies at rate <...
33 views

Cover time in Markov Chain from transition matrix

Given a process on a graph $X_{n} = \{x_{1}, ..., x_{n}\}$, is there a way to obtain the cover time, starting at any state $x_{i}$, from the transition matrix $\mathbf{P}$? I've obtained the expected ...
348 views

62 views

Asymptotic number of certain transition in a two-state Markov chain

Having a two-state Markov chain with the symmetric transition matrix: \begin{pmatrix} 1-p&p\\ p&1-p \end{pmatrix} The states are 1 and 2, let $n_{2\rightarrow1}(t)$ be the number of ...
Let $S_i$ be a Markov chain with transition Matrix $P$ $(0 ≤ p ≤ 1)$ \begin{equation*} P = \begin{pmatrix} 1-2p & 2 p& 0 \\ p & 1-2p & p \\ 0 & 2p & 1-2p \end{pmatrix} \...