1
vote
1answer
29 views

Proving Product of Transition Matrices is again a Transition Matrix.

Let $P = [p_{ij}]$ be an $n\times n$ transition matrix for an $n$-state markov chain. How do you prove that $P^2$, or even better, that $P^n$ is again a transition matrix? My approach leaves me ...
2
votes
1answer
28 views

Markov chains by hand

If I have a starting point: $A_T=[0,1]$ at $T=1$ and a one step transition matrix of: $B=\left[ \begin{align} &\frac34 & \frac14& \\& \frac1{20}& \frac {19}{20} &\end{align} ...
1
vote
0answers
18 views

Markov chains mixing time

Informally, the mixing time of a Markov chain is the time it takes to reach “nearly uniform” distribution from any arbitrary starting distribution. What does it mean by nearly uniform? I hope some one ...
0
votes
0answers
28 views

Perron Frobenius Theorem and Markov chains and more

I came across few ways of calculating convergence rates of Markov chains but I am a bit confused as to how these differ from each other and what may be the best way to calculate. The second ...
2
votes
1answer
61 views

Monotonicity and Convexity of Stochastic Matrices

The definition of stochastic monotonicity and convexity is given by "Stochastic Orders and Their Applications" by Moshe Shaked and George Shanthikumar (1994) as: Let $P = \{p_{i,j} \}$ be a ...
0
votes
3answers
85 views

How fast does this Markov chain converge?

Observe the above a Markov chain and limiting matrix of it. Finding the limiting matrix if it exists is easy but I am curious as to how fast this given matrix converges to its limiting matrix. Is ...
1
vote
0answers
12 views

Birkhoff-Neumann like result for stochastic matrices?

during my research I came along a nice lemma which looks like a Birkhoff-Neumann-theorem result, but in a version for stochastic matrices. Namely, I have: Lemma. Let $M$ be a stochastic matrix, then ...
0
votes
0answers
16 views

The second eigenvalue of a reducible stochastic matrix

The magnitude of the second dominant eigenvalue of a reducible matrix, as I know, is supposed to be 1, why it's not the case for this matrix : $$ \begin{matrix} 0 & 1 & 0 ...
5
votes
1answer
168 views

Generalization of the Jordan form for infinite matrices

Under what conditions is it the case that for a matrix $M$ whose rows and columns are indexed by a countably infinite set $S$ one has a Hamel basis consisting of generalized eigenvectors (i.e. $v \in ...
1
vote
0answers
50 views

iterates of generalized matrix system

This may be somewhat of an underspecified question but I'll nonetheless give it a try. In the context of applied work, I've recently come across systems of the form $$ ...
4
votes
1answer
72 views

How is the Chapman-Kolmogorov Equation not a fancy name for matrix multiplication?

The Chapman-Kolmogorov Equation: $$p^{m+n}(i,j)=\sum_kp^m(i,k)p^n(k,j)$$ Matrix Multiplication (with $[A]_{i,j}=a_{i,j}$ where $A$ is a linear map "" for B) $$[AB]_{i,j}=\sum_ka_{i,k}b_{k,j}$$ In ...
0
votes
1answer
63 views

Convergence rate of PageRank, the problem when the second eigenvalue is complex

As far as I know the Google matrix used to calculate the PageRank is not symetric, that means that some eigenvalues can be complex, furthermore, we know that the second eigenvalue is equal to the ...
2
votes
1answer
51 views

Inverse of a regular stochastic matrix

Is it true that the inverse of a regular stochastic matrix is also regular? Are there any other interesting features that the inverse may have of a regular stochastic matrix? Hope someone could answer ...
2
votes
0answers
30 views

Powers of (large) lower triangular matrix

Consider the following "game" of chance. Each time the player pushes a button he is awarded a random (finite, integer, non-negative) number of points. The probability of receiving any particular score ...
0
votes
0answers
41 views

Markov Transition Matrix

I have some data, shown below. How do I construct a transition matrix, for Markov Chain ? I need the formula to calculate observation data into transition matrix. Thanks! Accumulative ...
1
vote
0answers
33 views

markov chain computation

I consider a 2 state Markov chain: $X = \{1,2\}$, transitions are $M(i,j)$ and the matrix has a unique stationnary distribution $\pi$: $$ \pi(1) = \frac{M(2,1)}{2-M(1,1) - M(2,2)} \\ ...
1
vote
0answers
47 views

How's the damping factor in Google PageRank algorithm calculated

I'm doing some researches about Google's PageRank algorithm for my thesis, I've found that the damping factor x (for example), where x is in : P` = x.P + (1-x)Q where P is the original ...
0
votes
0answers
14 views

radius of markov chain

For an irreducible markov chain one can show, that $\limsup \sqrt[n]{p_{ij}^{(n)}}$ is independent of the choice of the states $i$ and $j$ where $p_{ij}^{(n)}$ is the probability to get from $x$ to ...
0
votes
2answers
61 views

Find steps to reach absorbing markov chain state

How can I find the steps it takes or days or whatever the time variable is till the matrix reaches the absorbing state. e.g. take the matrix (The probability of each column adds to 1) $$ \left[ ...
2
votes
0answers
32 views

markov spectral radius independent of states?

Let $\Pi$ be a stochastic matrix of an irreducible markov chain. We define the spectral radius of $\Pi$ as: $\rho(\Pi) := \limsup_{n \to \infty} \left( \pi^{(n)}_{(a,b)} \right)^{\frac{1}{n}}$ Why ...
1
vote
1answer
45 views

P is transition probability matrix.I is identitiy matrix.A is matrix whose entries are all 1.Then prove I+A-P is invertible

$P$ is the transition probability matrix for a finite irreducible markov chain. $I$ is identitiy matrix. $A$ is the matrix whose entries are all $1$. Prove $I+A-P$ is invertible. I don't have any ...
1
vote
2answers
95 views

Expected number of steps between states in a Markov Chain

Suppose I am given a state space $S=\{0,1,2,3\}$ with transition probability matrix $\mathbf{P}= \begin{bmatrix} \frac{2}{3} & \frac{1}{3} & 0 & 0 \\[0.3em] \frac{2}{3} ...
-1
votes
1answer
63 views

Proof that the square of a stochastic matrix is stochastic

We know that the square of a stochastic matrix is also stochastic, because the two-step transition matrix of a Markov chain is necessarily stochastic. However, in there another way to independently ...
0
votes
0answers
39 views

Combine transition and fertility matrices for youngest stage groups

I would like to ask how to combine the following information into the projection matrix. I do have data for transition (T) and fertility (F) stage matrices, so the projection matrix (A) is equal to ...
0
votes
0answers
70 views

A matrix of size $n\times n$ with several properties like Markov matrices

Could you find a square matrix $A=[a_{ij}]$ of size $n$ such that satisfies to following properties 1) For all $1\le i\le n$, $\sum_{j=1}^n a_{ij}=0$ 2) For all $i$, $a_{ii}<0$ and for $1\le i\ne ...
1
vote
1answer
125 views

Expected number of random binary vectors to make matrix of order n

I have the following problem: I pick random vectors from $\mathrm{F}_2^n$. The chance that position $i$ is $1$ equals $p_i$, $0$ otherwise (each position is picked independently). Let $X$ be a random ...
0
votes
0answers
52 views

Transition probability for time-homogeneous and inhomogeneous models

Consider the below matrices with four states - $0 , 1 , 2 , 3$ to be modelled by the means of a time-inhomogeneous discrete-time Markov chain. It's assumed the transition probabilities remain constant ...
1
vote
0answers
40 views

Finding Steady state using markov chains. Am I right?

Suppose that there are two doctors in a country town, Dr Black and Dr White. Each year, 13% of patients move from Dr Black to Dr White, while 19% of patients move from Dr White to Dr Black. Suppose ...
0
votes
1answer
56 views

Stochastic matrix with structure

Let $P \in [0,1]^{(n \times n)}$ be a stochastic matrix i.e $P_{ij} > 0 ~ \forall i,j$ and $\sum_{j}P_{ij} = 1~ \forall i$. Now let us impose additional structure on $P$ by saying that $P_{ij} + ...
2
votes
1answer
110 views

If $P$ is an invertible transition probability matrix, does $P^{-1}[i,j]$ have any interesting meaning?

Suppose we have a Markov chain transition probability matrix $P$ that is invertible, i.e., $P^{-1}$ exists. Question: Does there exist a meaningful interpretation of the $(i,j)$ entry in $P^{-1}$? ...
0
votes
0answers
41 views

When random walk Markov matrix of the graph is normal?

I'm considering random walk on undirected graph $G$. At every time step, walker moves to a random neighbour, with all neighbours being equally likely. With adjacency matrix $A$ the random walk Markov ...
1
vote
1answer
215 views

Reverse engineer transition matrix from steady state?

I had this open ended question. Is it possible to reverse engineer a transition matrix from the steady state. Is there a unique solution or could there be many? Basically given $P^{\infty}$ is there ...
1
vote
0answers
76 views

From Q matrix to Markov Chain

We are in the setting of a continuous time MC, as defined by Liggett in his book on continuous time markov processes, on a countable state space $S$. All of his MCs are defined on the space of right ...
2
votes
2answers
286 views

The second largest eigenvalue for Perron-Frobenius matrix

The Perron-Frobenius theorem is about the largest eigenvalue and eigenvector of a non-negative (irreducible) matrix. My question: Is there any estimation of the difference between the first and ...
0
votes
1answer
919 views

Finding the probability from a markov chain with transition matrix

Consider the Markov Chain with state space $S=\{v,w,x,y,z\}$, transition matrix below: $$\left[\begin{array}{cccccccccc} 0 & 0.4 & 0.6 & 0 & 0\\ 0 & 0.5 & 0.5 & 0 & ...
1
vote
0answers
100 views

Log Moment Generating function of a two-state Markov source

Let's say you have a two-state markovian source whose transition matrix is $P=\begin{pmatrix}1-\sigma & \sigma\\ \tau & 1-\tau\end{pmatrix}$, for the state 0 the data rate is 0 and for the ...
1
vote
0answers
38 views

What is the matrix norm in defining the generator of a continuous time Markov chain?

For a continuous time Markov chain with finite state space and Markov transition function $p(t)$, its generator $G$ can be defined entry-wise as $$ G_{i,j}:=\lim_{t\to 0^+} \frac{p_{i,j}(t) - ...
0
votes
1answer
128 views

Calculate the determinant of the matrix ad hence prove

By computing the determinant of $\lambda I-L$ where $L$ is the Leslie matrix, derive the Euler Lotka equation. $$L= \begin{bmatrix} b_{1} & b_{2} & \ldots & b_{w-1} & b_{w}\\ s_{1} ...
1
vote
2answers
469 views

How to create a transition matrix that will guarantee an outcome after infinite transitions

Let's assume we have the a transition matrix like: 0 0 0 1 2 0 2 4 0 3 6 0 4 7 2 5 9 3 6 6 6 7 7 7 8 8 8 9 9 9 First ...
1
vote
2answers
539 views

Perron-Frobenius theorem

In the proof of the Perron-Frobenius theorem why can we take a strictly positive eigenvector corresponding to the eigenvalue $1$? Before that, why can we even take a non-negative eigenvector? Books ...
4
votes
0answers
169 views

estimation of transition probabilities from aggregate data

Please, O mathematicians, help me understand the approach to the problem of estimating transition probabilities given only aggregate data in Kalbfleisch & Lawless' 1984 paper "Least-Squares ...
-2
votes
1answer
46 views

Continuous Markov chains and Q-matrices [closed]

Let $Q_1$ and $Q_2$ be the matrices for two continuous Markov chains and suppose there exists an invertible matrix $U$ such that $Q_1=U^{-1}Q_2U$. Show that $$e^{Q_1}=U^{-1}e^{Q_2}U$$
1
vote
1answer
167 views

Eigenvalues of a quasi-stochastic matrix

Quasi-stochastic In order not to make the title too long I used the term Quasi-Stochastic with this meaning: a quasi-stochastic matrix $Q$ is a square matrix $Q = ...
3
votes
2answers
350 views

How can I compare two Markov processes?

There is a discrete-time irreductible Markov process with $r$ possible states. $k$ observations were performed. At each observation a state of process was determined. $T_0 = \lbrace 0,1,\dots ...
1
vote
0answers
26 views

How many observations is the minimum?

I want to estimate model transition matrix for a process (Markov chain). How much observiations of state do I need? I would prefer this as a function dependent on $n$, where $n$ is number of possible ...
2
votes
1answer
3k views

How can I compare two matrices?

I have a matrice A. It is model probability matrice for some process (Markov chain). Then, I have estimated matrice B. I have to somehow compare these two matrices to tell whether process that gave ...
0
votes
1answer
2k views

Obtaining a two step transition matrix in a stationary Markov chain

I'm reading the chapter on Markov processes in DeGroot and do not find the explanation for the following thing: A transition matrix P is specified in the following way: $$P = \begin{pmatrix} 0.1 ...
2
votes
1answer
933 views

Computing the similarity between two matrices / Monte Carlo analysis

I am studying the article at the following link, http://www-stat.stanford.edu/~cgates/PERSI/papers/MCMCRev.pdf Which applies Monte Carlo analysis to a decryption problem. The math is admittedly over ...
1
vote
1answer
301 views

Transition matrix

I have a directed graph $G_1$. I extract its transition matrix $T_1$. Now I also have directed graph $G_2$, which is equal to $G_1$ with inverted edges. If I get its transition matrix $T_2$, what is ...
1
vote
3answers
205 views

The limit of matrices

Consider a square matrix $P$. We call it stochastic if it holds that $$ p_{ij}\geq0\text{ and } \sum\limits_{j=1}^m\,\,\,\,p_{ij} = 1 $$ for all $1\leq i,j\leq m$. I wonder when the following limit ...