0
votes
0answers
24 views

Markov Chain Geometric Convergence

Perhaps due to my non-mathematician background (lack of Measure Theory knowledge), I have some difficulties about the Markov Chain Theory. Given an ergodic (irreducible and aperiodic) Markov Chain ...
0
votes
0answers
24 views

Convergence of sequence of stationary distributions of Markov chains

I have a sequence of finite, discrete-time ergodic Markov chains indexed by a parameter $N$, and I want to prove that their stationary distributions are converging to a well-defined limit as $N\to ...
1
vote
4answers
168 views

Why do these expressions tend to zero?

In a chapter on Markov chains, it is claimed that $\binom {2n}{n} p^n(1-p)^n$ (where $p$ is a probability) tends to $0$ as $n$ tends to $\infty$. But why is this so? It is also claimed that for an ...
2
votes
0answers
92 views

Monotonic convergence of powers of a stochastic matrix

Let $P$ be a stochastic matrix (nonnegative and each row summing to 1). Assuming that $P^n$ converges to $\textbf{1}\pi$ as $n \rightarrow \infty$, where $\pi$ is a row vector (stationary distribution ...
3
votes
0answers
90 views

maximum renewal rate of a Markov chain

Consider a Markov chain $(X_t)$ on a state-space with a countably generated $\sigma$-algebra and assume this Markov chain allows for small sets of order one. This means there exist sets $\mathfrak{S}$ ...
2
votes
2answers
59 views

Convergence of an Ergodic process

I'm having trouble working through the math of the problem below. I believe the problem as described is an ergodic process. I've written a simple simulation of the problem, that converges to 66.6...6% ...
1
vote
1answer
410 views

Expected first hitting time in an absorbing Markov chain

There exists an absorbing $(M \times M)$Markov chain with the following transition matrix: $$ \begin{array}{ccccccc} p_{11} & p_{12}&0&\cdots&\cdots&\cdots&0\\ p_{21} & ...
2
votes
2answers
238 views

What does it mean for MCMC to converge?

I know that a Markov Chain is a discrete random process where the current state decides the next and in a random walk, the probability that we move from node u to v is 1/N(u). An MCMC sample will ...