1
vote
0answers
20 views

markov chain computation

I consider a 2 state Markov chain: $X = \{1,2\}$, transitions are $M(i,j)$ and the matrix has a unique stationnary distribution $\pi$: $$ \pi(1) = \frac{M(2,1)}{2-M(1,1) - M(2,2)} \\ ...
1
vote
0answers
26 views

How's the damping factor in Google PageRank algorithm calculated

I'm doing some researches about Google's PageRank algorithm for my thesis, I've found that the damping factor x (for example), where x is in : P` = x.P + (1-x)Q where P is the original ...
0
votes
0answers
11 views

radius of markov chain

For an irreducible markov chain one can show, that $\limsup \sqrt[n]{p_{ij}^{(n)}}$ is independent of the choice of the states $i$ and $j$ where $p_{ij}^{(n)}$ is the probability to get from $x$ to ...
0
votes
2answers
41 views

Find steps to reach absorbing markov chain state

How can I find the steps it takes or days or whatever the time variable is till the matrix reaches the absorbing state. e.g. take the matrix (The probability of each column adds to 1) $$ \left[ ...
2
votes
0answers
31 views

markov spectral radius independent of states?

Let $\Pi$ be a stochastic matrix of an irreducible markov chain. We define the spectral radius of $\Pi$ as: $\rho(\Pi) := \limsup_{n \to \infty} \left( \pi^{(n)}_{(a,b)} \right)^{\frac{1}{n}}$ Why ...
1
vote
1answer
44 views

P is transition probability matrix.I is identitiy matrix.A is matrix whose entries are all 1.Then prove I+A-P is invertible

$P$ is the transition probability matrix for a finite irreducible markov chain. $I$ is identitiy matrix. $A$ is the matrix whose entries are all $1$. Prove $I+A-P$ is invertible. I don't have any ...
1
vote
2answers
64 views

Expected number of steps between states in a Markov Chain

Suppose I am given a state space $S=\{0,1,2,3\}$ with transition probability matrix $\mathbf{P}= \begin{bmatrix} \frac{2}{3} & \frac{1}{3} & 0 & 0 \\[0.3em] \frac{2}{3} ...
-1
votes
1answer
54 views

Proof that the square of a stochastic matrix is stochastic

We know that the square of a stochastic matrix is also stochastic, because the two-step transition matrix of a Markov chain is necessarily stochastic. However, in there another way to independently ...
0
votes
0answers
26 views

Combine transition and fertility matrices for youngest stage groups

I would like to ask how to combine the following information into the projection matrix. I do have data for transition (T) and fertility (F) stage matrices, so the projection matrix (A) is equal to ...
1
vote
0answers
64 views

A matrix of size $n\times n$ with several properties like Markov matrices

Could you find a square matrix $A=[a_{ij}]$ of size $n$ such that satisfies to following properties 1) For all $1\le i\le n$, $\sum_{j=1}^n a_{ij}=0$ 2) For all $i$, $a_{ii}<0$ and for $1\le i\ne ...
1
vote
1answer
117 views

Expected number of random binary vectors to make matrix of order n

I have the following problem: I pick random vectors from $\mathrm{F}_2^n$. The chance that position $i$ is $1$ equals $p_i$, $0$ otherwise (each position is picked independently). Let $X$ be a random ...
0
votes
0answers
37 views

Transition probability for time-homogeneous and inhomogeneous models

Consider the below matrices with four states - $0 , 1 , 2 , 3$ to be modelled by the means of a time-inhomogeneous discrete-time Markov chain. It's assumed the transition probabilities remain constant ...
1
vote
0answers
29 views

Finding Steady state using markov chains. Am I right?

Suppose that there are two doctors in a country town, Dr Black and Dr White. Each year, 13% of patients move from Dr Black to Dr White, while 19% of patients move from Dr White to Dr Black. Suppose ...
0
votes
1answer
53 views

Stochastic matrix with structure

Let $P \in [0,1]^{(n \times n)}$ be a stochastic matrix i.e $P_{ij} > 0 ~ \forall i,j$ and $\sum_{j}P_{ij} = 1~ \forall i$. Now let us impose additional structure on $P$ by saying that $P_{ij} + ...
2
votes
1answer
93 views

If $P$ is an invertible transition probability matrix, does $P^{-1}[i,j]$ have any interesting meaning?

Suppose we have a Markov chain transition probability matrix $P$ that is invertible, i.e., $P^{-1}$ exists. Question: Does there exist a meaningful interpretation of the $(i,j)$ entry in $P^{-1}$? ...
0
votes
0answers
39 views

When random walk Markov matrix of the graph is normal?

I'm considering random walk on undirected graph $G$. At every time step, walker moves to a random neighbour, with all neighbours being equally likely. With adjacency matrix $A$ the random walk Markov ...
1
vote
1answer
182 views

Reverse engineer transition matrix from steady state?

I had this open ended question. Is it possible to reverse engineer a transition matrix from the steady state. Is there a unique solution or could there be many? Basically given $P^{\infty}$ is there ...
1
vote
0answers
72 views

From Q matrix to Markov Chain

We are in the setting of a continuous time MC, as defined by Liggett in his book on continuous time markov processes, on a countable state space $S$. All of his MCs are defined on the space of right ...
2
votes
2answers
253 views

The second largest eigenvalue for Perron-Frobenius matrix

The Perron-Frobenius theorem is about the largest eigenvalue and eigenvector of a non-negative (irreducible) matrix. My question: Is there any estimation of the difference between the first and ...
0
votes
1answer
820 views

Finding the probability from a markov chain with transition matrix

Consider the Markov Chain with state space $S=\{v,w,x,y,z\}$, transition matrix below: $$\left[\begin{array}{cccccccccc} 0 & 0.4 & 0.6 & 0 & 0\\ 0 & 0.5 & 0.5 & 0 & ...
1
vote
0answers
95 views

Log Moment Generating function of a two-state Markov source

Let's say you have a two-state markovian source whose transition matrix is $P=\begin{pmatrix}1-\sigma & \sigma\\ \tau & 1-\tau\end{pmatrix}$, for the state 0 the data rate is 0 and for the ...
1
vote
0answers
33 views

What is the matrix norm in defining the generator of a continuous time Markov chain?

For a continuous time Markov chain with finite state space and Markov transition function $p(t)$, its generator $G$ can be defined entry-wise as $$ G_{i,j}:=\lim_{t\to 0^+} \frac{p_{i,j}(t) - ...
0
votes
1answer
124 views

Calculate the determinant of the matrix ad hence prove

By computing the determinant of $\lambda I-L$ where $L$ is the Leslie matrix, derive the Euler Lotka equation. $$L= \begin{bmatrix} b_{1} & b_{2} & \ldots & b_{w-1} & b_{w}\\ s_{1} ...
1
vote
2answers
449 views

How to create a transition matrix that will guarantee an outcome after infinite transitions

Let's assume we have the a transition matrix like: 0 0 0 1 2 0 2 4 0 3 6 0 4 7 2 5 9 3 6 6 6 7 7 7 8 8 8 9 9 9 First ...
1
vote
2answers
445 views

Perron-Frobenius theorem

In the proof of the Perron-Frobenius theorem why can we take a strictly positive eigenvector corresponding to the eigenvalue $1$? Before that, why can we even take a non-negative eigenvector? Books ...
1
vote
0answers
56 views

Lower bound on the minimum entry of the stationary distribution

Let $\phi$ be the stationary distribution vector (Perron vector) arrising from a stochastic (not doubly stochastic) matrix $P$. Can you think of a way to lower-bound $\min\phi$ in terms of $\parallel ...
4
votes
0answers
161 views

estimation of transition probabilities from aggregate data

Please, O mathematicians, help me understand the approach to the problem of estimating transition probabilities given only aggregate data in Kalbfleisch & Lawless' 1984 paper "Least-Squares ...
-2
votes
1answer
43 views

Continuous Markov chains and Q-matrices [closed]

Let $Q_1$ and $Q_2$ be the matrices for two continuous Markov chains and suppose there exists an invertible matrix $U$ such that $Q_1=U^{-1}Q_2U$. Show that $$e^{Q_1}=U^{-1}e^{Q_2}U$$
1
vote
1answer
158 views

Eigenvalues of a quasi-stochastic matrix

Quasi-stochastic In order not to make the title too long I used the term Quasi-Stochastic with this meaning: a quasi-stochastic matrix $Q$ is a square matrix $Q = ...
3
votes
2answers
325 views

How can I compare two Markov processes?

There is a discrete-time irreductible Markov process with $r$ possible states. $k$ observations were performed. At each observation a state of process was determined. $T_0 = \lbrace 0,1,\dots ...
1
vote
0answers
21 views

How many observations is the minimum?

I want to estimate model transition matrix for a process (Markov chain). How much observiations of state do I need? I would prefer this as a function dependant of n, where n is number of possible ...
2
votes
1answer
2k views

How can I compare two matrices?

I have a matrice A. It is model probability matrice for some process (Markov chain). Then, I have estimated matrice B. I have to somehow compare these two matrices to tell whether process that gave ...
0
votes
1answer
1k views

Obtaining a two step transition matrix in a stationary Markov chain

I'm reading the chapter on Markov processes in DeGroot and do not find the explanation for the following thing: A transition matrix P is specified in the following way: $$P = \begin{pmatrix} 0.1 ...
2
votes
1answer
923 views

Computing the similarity between two matrices / Monte Carlo analysis

I am studying the article at the following link, http://www-stat.stanford.edu/~cgates/PERSI/papers/MCMCRev.pdf Which applies Monte Carlo analysis to a decryption problem. The math is admittedly over ...
1
vote
1answer
290 views

Transition matrix

I have a directed graph $G_1$. I extract its transition matrix $T_1$. Now I also have directed graph $G_2$, which is equal to $G_1$ with inverted edges. If I get its transition matrix $T_2$, what is ...
1
vote
3answers
202 views

The limit of matrices

Consider a square matrix $P$. We call it stochastic if it holds that $$ p_{ij}\geq0\text{ and } \sum\limits_{j=1}^m\,\,\,\,p_{ij} = 1 $$ for all $1\leq i,j\leq m$. I wonder when the following limit ...