# Questions tagged [markov-chains]

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

3,607 questions
1answer
12 views

### Krylov Bogolubov Theorem in Unbounded space

Let $P$ be a Feller transition probability on an unbounded space $X$, if there exists $x\in X$ such that the sequence $\{P^n(x, \cdot)\}_{n\ge 0}$ is tight, then show that there is a probability ...
0answers
17 views

### understanding time-homogeneous markov chain

Could anyone make me understand the definition here 1 on page 7 definition 2.25, I quite do not understand the notation $P(a)(A)$, what does this mean? Also, is $P(a, A)$ a probability measure from ...
0answers
20 views

### How to show that discrete-time Markov chain run backwards in time is again a Markov chain [on hold]

I have a homework question on Markov Chains, can someone please give some help. The full question is: Show that a discrete-time Markov chain run backwards in time (from some time n and state i) is ...
1answer
20 views

0answers
18 views

### understanding transitional probabilities and measure

In this example 1.11, 1, could anyone just explain to me the way he has defined the $\mathcal P(x,\cdot)$? I know the definition of $\delta_x$ which indicates the probability centered at point mass. ...
1answer
18 views

0answers
11 views

### If a homogenous transition matrix has some entries, which are unknown, then must they be unique?

If a homogenous transition matrix has some entries, which are unknown, then must they be unique? That is if one's given a matrix, where: some elements are known some are unknown and marked e.g. "?" ...
0answers
52 views

### Is the matrix $\sum_{g \in G} a_g \rho(g)$ normal and what further properties does it have?

Let $\rho$ be the regular representation of $G$. $S \subset G$ a generating set, $|g| := |g|_S=$ word length with respect to $S$. Then I construct such a matrix, where we have some ordering $g_i$ of ...
1answer
38 views

### How to determine the transition matrix of the markov chain $X_n = f(X_{n-1}, \xi_n), n \geq 0$

I am having trouble to find the transition matrix of the following question: Let $X_0$ be a random variable taking values in a countable set $I \subset \mathbb{R}$. Let $(\xi)_{n \geq 0}$ be a ...
0answers
21 views

1answer
20 views

### Show that $\sum^{N-1}_{i=1} \frac{N-1}{i(N-i)}$ is approximately $2\log N$ for large $N$.

I am working on a problem relating to a Markov chain, that results in the sum: $$\sum^{N-1}_{i=1} \frac{N-1}{i(N-i)}$$ For the expected time until reaching the $N^{th}$ state from $0$. The ...
1answer
32 views

0answers
24 views

### Are the terms Limiting distribution and Stationary distribution properly perceived?

Consider a Markov chain $(X_n)_n$ on $S=\{1, 2\}$ with initial distribution $α$ and the transition matrix $P = \begin{bmatrix} 2/3 & 1/3 \\ 2/3 & 1/3 \\ \end{bmatrix}$ Limiting ...