# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

62 views

45 views

### Normalisation of an invariant measure

Is there an example of a Markov chain with invariant measure $\pi$ and $\sum_{i \in I}\pi(i) = \infty$ that can be normalised so that we can consider an invariant distribution instead? This is a ...
25 views

49 views

### How many stationary distributions does a time homogeneous Markov chain have?

I've been given the following definition: For a THMC with one step transition matrix $\mathbf{P}$, the row vector $\mathbf{\pi}$ with elements $(\pi_{i})_{i \in S}$ (where $S$ is the state space) ...
28 views

35 views

### Invariant measure nomenclature

I'm looking through my notes and I've come across the following line: If $\sum_{i \in I}\pi(i) = \infty$ then we (usually) say that the Markov chain doesn't have an invariant distribution. My ...
18 views

### DTMC and CMTC , First Passage Probabilities and Expectation.

Let the sample space $S=(0,1,2)$. Let $Q=\begin{pmatrix} -4 & 2 & 2 \\ 3 & -5 & 2 \\ 0 & 3 & -3 \\ \end{pmatrix}$ be the generator of our CTMC. ...
45 views

### Markov chain matrix multiplication - Finding all path probability in a graph

I realize there are many Markov Chain questions on this site. I have reviewed all relevant questions, the closest to my question are: Finding the probability from a markov chain with transition matrix ...
43 views

Markov Chains: Finding the Embedded DTMC (transition probability matrix) $P(t)$ from generator matrix $Q$ where the sample space $S=(0,1,2)$ $Q=\begin{pmatrix} -4 & 2 & 2 \\ ... 1answer 17 views ### How many stationary measures does a random walk with absorbing barriers have? Suppose I have a markov chain with finite state space$0,\ldots, N$. At each state$1, \ldots, N-1$, we have that the probability of going up and down one state is of probability$\frac{1}{2}$. Now, ... 1answer 22 views ### What is an example of a Markov Chain with two stationary measures? I am trying to come up with a transition matrix for a Markov Chain with two stationary measures, but am not able to construct it. Would anyone have an example? Thanks. 1answer 21 views ### Stochastic Kernel almost surely determined by semidirect product? Given a measurable space$(\Omega, \mathcal{F})$with two probability measures$\mathbb{P}_1$,$\mathbb{P}_2$and a second measurable space$(X,\mathcal{A})$with two stochastic kernels$\mu_1, \mu_2$... 0answers 18 views ### marginalised markov chain If there is a Markov chain for the joint variable$z=(x,y)$, the marginal process$x$is not, in general, Markovian itself. However, if we consider the probability of a two time step process $$p(z_0,... 0answers 42 views ### What is the invariance principle of Random Walks? Several papers I have read allude to the fact that a random walk is invariant; however, I have been unable to find any reference to support this fact. Could anyone explain why random walks are ... 0answers 52 views ### Finite irreducible Markov chain The question I have is stated as follows: Show that for any finite-state irreducible Markov chain$$\max_{i,j}\mathbb E_iT_j\le C$$where the constant$C$only depends on the number of states and$\...
I'm wanting to sample the discrete uniform distribution over $n = 10$ integers using MCMC. My question concerns the transition probability matrix, $P$. As I understand it, any symmetric, irreducible ...