# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

125 views

### Markov chain from Poisson

Let $K_t$ be a Poisson process with rate $1$ and $X_n=K_n-n$ $, \ \ \ n\in \mathbb{N}$ am asked to determine whether it is null or positive recurrent, we already know it is recurrent. I ...
29 views

Broken disk head: We would like to read 1 byte = sequence of 8 bits from a disk, starting from bit 0. Our disk head reads 1 bit at a time. Disk head can only move forward, but after reaching bit 7 ...
84 views

### Is the following interpretation for the stationary distribution of a Markov process correct?

Imagine I have some Markov process with stationary distribution $\pi$ and a mixing time of $\tau$ after which $|Prob[x=s_i] - \pi(s_i)| \leq \epsilon$. Can I assume the following: A state $(x=s_i)$ ...
21 views

### Problem with the uniform transience

Let $X$ be a Borel space and let us consider a Markov Chain $(\Phi_n)_{n\geq 0}$ on this space given by the stochastic kernel $$P(x,\mathrm dy) = p(x,y)\mu(\mathrm dy)$$ where the density $p$ is ...
333 views

### In finite-state Markov chain state $i$ is transient

Can you help me please with proof of this question: Prove, that in finite-state Markov chain state $i$ is transient if and only if is exist state $k$ such that $i\rightarrow k$ but k $\nrightarrow i$....
240 views

10 views

### Filtering/MCMC methods for this HMM

I have a Discrete HMM with hidden Markovian signals of the form $\{X_t\}_{t \in [0, \infty)} \in \{ 1,2,3\}$ and observed outputs of the form $\{Y_n\}_{n \in \mathbb{N}} \in \{ 1,2\}$. Each ...
33 views

### Better to go first in Snakes and Ladders?

We consider the game as described in http://www.datagenetics.com/blog/november12011/ . Each person rolls a dice and the person who gets 6 on the face can start and the other keeps waiting. If the ...
92 views

### Probability of losing lotteries needed

A person earns $x_i$ amount of money every month where $x_i$ is an exponential random variable with parameter $\lambda_1$. The amount $x_i(1-p)y$, here $0 \leq p\leq 1$ and $y$ is exponentially ...
30 views

### Statement of the strong Markov property in Norris' book

In J.R.Norris' Markov chains book, the strong Markov property for discrete-time, Markov chains is stated and proved as follows: Let $(X_n)_{n \geqslant 0}$ be a Markov chain with transition ...
22 views

### Perron-Frobenius Theorem: Markov Chain -> Matrices

I am interested in finding out a way how to transform the stochastic results of perron-frobenius for markov chains to any matrix. I am aware that perron-frobenius was originally proofed with linear ...
29 views

### Series with Markov Chains Probabilities

Notation For each $t \in \mathbb{N}$, let $h_t \in H$ be a random variable that follows a Markov chain, and $h^t \equiv \{h_0,h_1,\dots,h_t\} \in H^t$. Let $\Pi(h^{t})$ be the probability that a ...
29 views

44 views

### For a finite state irreducible aperiodic MC, show that $P^{d^2}$ has all coordinates positive.

Suppose $X_n$ is an irreducible aperiodic finite state MC, with $P$ being the transition matrix. Then we know that $P^n$ has all positive entries for some $n\in\mathbb N$. If the state space $S$ of ...
25 views

### How to test quality of probability estimates?

I have a Markov chain model which produces a probability distribution for absorption in 4 possible absorbing states. I.e. the model estimates the probability distribution for a discrete random ...
58 views

### Prediction Interval from Markov Chains

Thank you for taking the time to look at my question. Short, less involved question: How do you find Prediction Intervals with non-Gaussian residuals? Here is the situation: I have made a model that ...
24 views

### When can an embedded Markov chain X for a Markov process Y be reducible?

It's pretty widely documented that a Markov process Y is reducible/irreducible if and only if the embedded Markov chain X is reducible/irreducible. However I'm not sure this works in reverse. I'm ...
20 views

30 views

18 views

34 views

### Simple random walk on the $N$-cycle

I am considering the following example: In my lecture notes we noted that "the functions $(\phi_j)_j$ form a basis". I think they refer to the space $\mathbb{C}^G$ where $G$ is the above ...
44 views

### Markov chain of transition probabilities

Let $P$ be a transition matrix on a discrete state space with $N$ elements. $P_{i,j}$ is the probability of going from state $i$ to state $j$. Let $\pi$ be the stationary distribution. Let $\{X_n\}$ ...
45 views

### Birth-Death process with shifted exponential distribution

In the general framework of $M/M/1$ queue we have rate $\lambda$ and an exponential service time $\mu$, we can set up the transition rate matrix intuitively. However, if the service times satisfy ...
25 views

### Understanding a Markov decision process

We have an insect that is resting on a vertex of a square at each point of time $t=0,1,2..$. The vertices are labelled from 1 to 4. 1 is given to the lower left vertex, 3 to the upper left vertex, 2 ...
54 views

### Finding a stationary distribution for a Markov chain with a binomially distributed transition matrix

The distribution isn't exactly binomial.. I'll explain what I mean: I am considering a Markov chain with N+1 states (denoted by $0,1,...,N$), the probability of going from state $i$ to state $j$ is ...
19 views

### convergence of markov chain, find distribution of limit

let $T=\mathbb{N}$ and $(X_t)_{t \in T}$ be a markov chain with state space $E=\{a,b\}$. the one step transition matrices are given by \begin{equation*} P(t,t+1)= \begin{pmatrix} 0,2+0,8f(t) &...
Could anyone help me with this? Let $X_n$ be an irreducible Markov chain on the state space $S=\{1,\ldots,N\}$. Show that there exist $C<\infty$ and $\rho <1$ such that for any states $i,j$, \$...