# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

37 views

### Definition of Past in Markov Property

Usually in textbooks, the definition of the Markov Property reads as: P(X_n\leqslant x_n|X_{n-1}\leqslant x_{n-1},...,X_{0}\leqslant x_{0})=P(X_n\leqslant x_n|X_{n-1}\leqslant x_{n-1}...
37 views

### A cycle of an irreducible Markov chain

Suppose we have a discrete-time Markov chain with $n$ states, with transition matrix $P$. The definition of irreducibility tells us that for any two states $x$ and $y$, there exists an integer $t$ ...
60 views

65 views

46 views

### Markov processes: Hitting times for a point form an i.i.d. sequence

Say that I have a recurrent time-homogenous diffusion process X (continuous strong markov process) and two points $x,y$. If $X_t$ goes from $y$, to $x$ and then back to $y$ again we denote it as a "...
21 views

### a conceptual question on markov chain [duplicate]

Suppose $\{X_n,n\ge 0\}$ and $\{Y_n,n\ge0\}$ are two independent discrete-time markov chains (DTMC) with state space $S=\{0,1,2,\ldots\}$. Prove or give a counterexample to: $\{X_n+Y_n,n\ge 0\}$ is ...
28 views

### How to Compute Distributions in Hidden Markov Models?

Let $Z_1, Z_2, ..., Z_n$ be the latent variables, and $X_1, X_2, ... X_n$ be the observed ones in a hidden Markov models. Let's assume that the parameters of the hidden Markov models are known: the ...
73 views

### Error in Billingsley?

Problem 8.25 in the third edition of Probability and Measure by Billingsley (1995, p. 142) is as follows: Suppose that an irreducible [Markov] chain of period $t>1$ has a stationary ...
29 views

### Difference between AR model and Markov Chain

We know that Markov Chain can be represented as $$x_t=ax_{t-1}+\epsilon_t,$$ where $\{x_t\}$ are states, $\epsilon$ is noise, and $a\neq 0 \in R$. For the AR model, we know that, the first order ...
71 views

### How many steps would it take to get to the top of this staircase?

There are 26 steps in a staircase. You have a 51% chance to step onto the next step, and a 49% chance to step back down to the step prior. Assuming you are already on the first step, how many steps ...
22 views

### Find $P(\eta_t=m)$, $m=0,1,2,\dots,$

Let $\epsilon_t$, $t=1,2,\dots$ independent random variables with $P(\epsilon_t=1)=p$ and $P(\epsilon_t=-1)=1-p$. If $\eta_0=0,\eta_t=\eta_{t-1}+\epsilon_t$ , $t=1,2,\dots$ where $\eta_t$ is ...
114 views

### Ergodic components of Markov chain by transition matrix

I would like to find an algorithm for obtaining all ergodic components of a finite Markov chain with discrete time defined by its transition matrix (i.e. ergodic subchains into which the given chain ...
28 views

### Can an irreducible, recurrent continuous time Markov chain have spontaneous states?

Let $(X_t)_{t\geq0}$ be a continuous time Markov chain on some (possibly countably infinite) state space $S$ with Q-Matrix $q(\cdot,\cdot)$, transition function $p_t(\cdot, \cdot)$ and invariant ...
94 views

### Combinatorics Statistics Probability of a Letter Chain

What formula could help me quantify the probability of a chain of three letters (English Alphabet) where each letter is based on the previous one (stochastic modeling, Markov-chains, probabilities) ...
21 views

### Metropolis Markov Chain and Mixing Time

I have a statistical mechanical system, which I would like to sample with the Gibbs distribution using a Metropolis-Markov chain. I think the following are standard questions, but I am not sure what ...
77 views

11 views

### The restriction of the green function is non-degenerate

On the context of irreducible Markov chains on a finite graph one defines $$G = \sum_{k=0}^\infty \hat{Q}^k$$ where $\hat{Q}$ is the restriction of a stochastic matrix to a subset (one can think ...
30 views

### What is the practical meaning of probability vectors?

I have been reading a lot about probability vectors, as a part of "Introduction to Probability" course. Now, whenever it was mentioned, it was defined theoretically as a vector whose entries add up to ...
87 views

### Weak convergence of a sequence of stationary distributions to another stationary distribution

Let $\{X_n(t) \in \mathbb{R}^+\}$ for each $t \in (0,1)$ denote a discrete time Markov chain (with time index $n$ and parameterized by $t$). For each $t$, the Markov chain $\{X_n(t)\}$ has a unique ...
I know that Markov matrices have steady state since they always have eigenvalue $\lambda = 1$. We just solve the system of equations $A\vec x = 1 \cdot \vec x$ or \begin{cases} k_{a\to a} a + k_{b\...