# Questions tagged [markov-chains]

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

4,288 questions
Filter by
Sorted by
Tagged with
6 views

20 views

12 views

### Using Chapman-Kolmogorov Property to prove v=Qv

How would you use the Chapman-Kolmogorov property ($Q_{t+s}=Q_tQ_s$) to prove that v (a column vector distribution over the sample space) is a stationary distribution of Markov Chain $X_t$ with ...
17 views

### Stationary Distribution of Ehrenfest Markov Chain

The example in my book for an Ehrenfest Markov chain is: A system of of two urns, A & B where there are 2n balls total in both urns. We are assuming that there are $i$ balls in urn A and $2n - i$ ...
10 views

### Expected hitting times for simple random walk on a hypercube

Setup In an $n$-dimensional hypercube $C_n = \{0,1\}^n$, we define the Hamming distance of two vertices $d(A,B)$ to be the number of coordinates in which they differ. (e.g. $d((0,0,1), (1,0,1)) = 2$.)...
11 views

32 views

35 views

16 views

### Recurrent Harris Chain

A Harris chain, $X_n$, as defined in Durrett, Probability, sec. 5.8 of the V ed., depends on the definition of two measurable sets A and B and some probability measure $\rho$ (details are given on ...
31 views

### Could indicator function be introduced into probability?

Suppose there are 3 random variables: $A_t\in\{0,1\},B_t,\text{and}\; B_{t-1}$. I would like to write a joint probability distribution to express the idea: At time step $t$, $B_{t-1}=b$ has been ...
15 views

13 views

### Non-linear Markov chain model [duplicate]

Could anyone explain to me this Markov chain model? $$S_{k+1}= P(S_k+S_k^0).$$ Please allow me to give a link from the paper I was read this equation $(6)$ here https://drive.google.com/file/d/...
12 views

### Markov chain connected with recurrent events

I am reading Feller Volume 1, and this example is in page 382. I understand that $f_1= q_1$ and $f_2 = p_1 q_2$, but I don't understand how to derive $p_k$ in general (which I highlight with the ...
This is What I have so far and I am not sure if I am on the right track: Consider the matrix Q = $\left[ \begin{matrix} λ & 0 \\ 0 & μ\\ \end{matrix} \right]$ So then the ...
### methods for finding the limit of $n$-step probability
I have the following transition matrix of a markov chain: \mathbb{P}= \begin{bmatrix} 1 & 0 & 0 & 0 & 0\\ q & \delta & p & 0 & 0\\ 0 & q & \delta & p &...