# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

54 views

### In a transition matrix, is the biggest component of the stationary distribution the one that correspond to the column with the biggest entries-sum?

Given a transition matrix, is the biggest component of the stationary distribution the one that correspond to the column whose sum of entries is the biggest among all columns? (By "correspond" I mean ...
3k views

### What is the eigenvalue of stochastic matrix?

I have a stochastic matrix $A \in R^{n \times n}$ whose sum of the entries in each row is 1. When I found out the eigenvalues and eigenvectors for this stochastic matrix, it always happens that one of ...
29 views

### Soft Question - book recommendation - Stochastic Processes

My mother language book on stochastic processes is pretty much complete(~500 pages) but would like another one in English, to have in my library. I'm looking for a similar book containing the ...
43 views

### Why are the $n$-step transition probabilities well defined?

I was reading a proof for the Chapman-Kolmogorov equations and now I understand why it is the case that for a discrete-time homogeneous Markov Chain $X=(X_n) _{n\geq 0}$ (with state space $S$) the ...
29 views

### Up-to-date or Behind - [Markov Chain]

Alex is taking a bioinformatics class and in each week he can be either up-to-date or he may have fallen behind. If he is up-to-date in a given week, the probability that he will be up-to-date (or ...
26 views

### Random Early Discard Markov Chain

I'm trying to sketch the Markov chain for a Random Early Discard queueing policy where customers arrive to the queue of infinite size according to a Poisson process with rate $\lambda$. Customers that ...
36 views

### How to find the limit of a matrix $P^n = UD^nU^{-1}$ where $D$ is a diagonal matrix of eigenvalues and $U$ a matrix of eigenvectors?

If we have a matrix where $P = UDU^{-1}$, where $D$ is a diagonal matrix of real eigenvalues that are less than or equal to 1, and $U$ is the corresponding matrix of eigenvectors, how can we show that ...
54 views

Ok so we are given a Markov chain $X_n$, $P=P(ij)$ as the transition matrix and the $(\pi_1,\pi_2,\pi_3,...,\pi_n)$ as steady-state distribution of the chain. We are asked to prove that for every $i$: ...
24 views

### Detailed balance implies time reversibility, how about the converse?

Given a Markov chain (finite state space) $X_1,X_2,...$ with transition matrix $P$ and initial distribution $\pi$, if they satisfy $\pi(x)P(x,y)=\pi(y)P(y,x)$, we say they satisfy detailed balance. ...
50 views

### Markov Chain: memoryless property?

I have a question about the Markov Chain. We were doing derivation of the Chapman-Kolmogorov Equations, the $n+m$ step state transition probability (please see below): where $P_{i,j}$ is the ...
41 views

### Has a Markov chain in compact metric space a stationary distribution (possibly non-unique)?

Let $Y_n$ be a Markov chain in a compact metric space. Is it true that it has a stationary distribution?
45 views

### Showing that a Markov-Chain has this property.

So I have to show that: Let $\{X_n\}$ be a Markov chain. Show that the property $P(X_{n+1}=i|X_n=j_n,X_0=j_0)=P(X_{n+1}=i|X_n=j_n)$ holds. Hint: use the Markov-property. The Markov Property being: ...
58 views

### I think I've found an invariant distribution for a transient discrete Markov chain - Where is my mistake?

Let $E$ be an at most countable set equipped with the discrete topology $\mathcal E$ $X=(X_n)_{n\in\mathbb N_0}$ be a discrete Markov chain with values in $(E,\mathcal E)$, distributions ...
31 views

### Meaning of $\bigvee$ for collection of sets

In a book* I found the following: $\mathcal{F} = \bigvee_i \mathfrak{B}(X_i)$ where $\mathcal{F}$ denotes a $\sigma$-algebra on a markov chain and $\mathfrak{B}(X_i)$ is the Borel sigma algebra ...
49 views

321 views

### Understanding detailed balance equations

I'm trying to understand how the equilibrium distribution satisfy the detailed balance equation. To my understanding, I only understand that a detailed balance equation would only be satisfied if ...
163 views

### How fast does this Markov chain converge?

Observe the above a Markov chain and limiting matrix of it. Finding the limiting matrix if it exists is easy but I am curious as to how fast this given matrix converges to its limiting matrix. Is ...
84 views

### Continuous time Markov chain. proportion of time spent in state i

If a question asks for the proportion of time spent in a specific state is this the same as the stationary distribution or something else? For continuous time Markov chain with finite state space.
107 views

### Weather Transition Matrix

Based on observation, I've gathered some data: Day 1 2 3 4 5 6 7 8 9 10 S R S F S R F F R S (S) = Sunny (R) = Rainy (F) = Foggy How do I construct this into a transition matrix, for markov ...
66 views

### Markov chain probability that a state changes

For the Markov chain given below what is the best way to find the probability P{$x_n = x_{n-1}$} and P{$x_n \neq x_{n-1}$} The transition matrix of the chain is \begin{array}{} 1/3 & 2/3 & ...
664 views

### Random walk with single absorbing boundary

There is a random walk on a linear lattice of size $\{0,N\}$, where $0$ is the origin and a reflecting boundary and $N$ is the absorbing boundary. It moves forward or backward one step at a time with ...
120 views

### Stationary distribution for a Markov chain which is not irreducible

I have a Markov chain with $K$ states $S$: {$s_1,s_2,...,s_K$}. $s_1$ is reachable from any state in $S$; however not all the states can be reached from $s_1$. What does the stationary distribution ...
352 views

### Load balance equation of birth-death process

I am not able to understand the reasoning behind the load-balance equation of the birth-death processes (Markov chain) . The equation is $\pi_ib_i = \pi_{i+1}d_{i+1}$ , where the symbol have their ...
174 views

### Expected length of generating a pattern (throwing dice)

A fair die is tossed repeatedly. The experiment ends as soon as the last six outcomes form the pattern 131131 What is the expected length (i.e. the number of rolls of the die) of this experiment?
66 views

### Kolmogorov backward equation: what is it computing?

The kolmogorov backward equation states: $P_{ij}^{'}(t) = \sum_{k \ne i} q_{ik}P_{kj}(t) - v_iP_{ij}(t)$ Is this computing the rate of transition from i to j?
1k views

### Markov chain: closed, finite classes are recurrent?

In Norris: Markov Chains the closed class C is defined as one for which $i\in C$ and $P_i(X_n=j \text{ for some }n\ge0)>0$ implies that $j\in C$. Here's theorem 1.5.6 from the book with proof ...
151 views

### Need help in understanding state transition diagram of a convolutional coder. How are the output bits calculated?

Have a look at the above figure. I am confused in how the output bits are calculated. e.g. according to my understanding a state transition from 00 to 10 (with input bit 1) should produce output 10 ...
120 views

### Discrete Markov Transition Matrix

Well, I've been reading over the internet but I've been unable to find a straight answer. I've got a transition matrix for a Markov Discrete Chain. I've made the graph and according to my knowledge, ...
106 views

### Expected number of shots for a game to end

A basketball player plays a shooting game. He gets +1 point if he scores a basket and -2 points if he misses. He starts with 0 points. The game ends when the player reaches +10 or -10. What is the ...
Suppose I have the following equilibrium probability distribution: $\vec π = ({2\over5} , {1\over5} , {3\over20},{1\over4})$, corresponding to states 0,1,2,3, respectively.From my possible states of ...
By using the memoryless property, I find the CDF of sojourn time of a CTMC as follows : $$F_X(t) = 1-e^{-F_X'(0)t}$$ I am slightly confused about the intuitive meaning of the term $F_X'(0)$. How ...