# Questions tagged [markov-chains]

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

5,991 questions
Filter by
Sorted by
Tagged with
1 vote
37 views

• 3,584
1 vote
84 views

• 1,054
99 views

### Random walk on the edges of a square where staying at same position is allowed

The classic question of finding the expected number of steps to move from one corner of a square to the opposite. I found an intuitive way to do it by considering that we are at position $(0,0)$ and ...
• 301
23 views

### How To Prove That The Absolute Convergence Of The One-Step Transition Matrix?

I am trying to prove that the n-step transition matrix in a Markov Chain satisfies the condition that the sum of each row is one (ie, each row is a valid probability distribution). I have done this ...
• 185
31 views

• 603
1 vote
46 views

### Are Markov chains $X_{t+r}$, $X_{2t}$ and $(X_t; X_{t+1})$?

I'm trying to solve a problem. Let $X_n$ be a Markov chain with values in the set $H$. Will there be Markov chains $X_{n+r}$, where $r > 0$ is fixed, $X_{2n}$, $(X_n, X_{n+1}) \in H^2$? The first ...
• 133
1 vote
34 views

### Prove: every strongly connected graph has at most one stationary distribution without manipulating with eigenvector

In mcs.pdf, Problem 21.12 says: A digraph is strongly connected iff there is a directed path between every pair of distinct vertices. In this problem we consider a finite random walk graph that is ...
• 416