# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

20 views

### Example of Markov property

I am reading Durrett's Probability: Theory and Examples, and trying to understand its context about Markov Chains. It is not hard to understand the theorem and proofs but when it comes to concrete ...
34 views

### proof of Markov chain Monte Carlo

This is the first step of proof of MCMC in my notes I have a question, how come $\pi(x)\pi(x_p\mid x)=\pi(x_p)\pi(x\mid x_p)$? Is it true for any markov chains which are ergodic and aperiodic? The ...
50 views

### confused about proposal distribution in MCMC

This is a question from notes I have some questions regarding the proposal distribution which is $N(x,1)$ Is the proposal distribution symmetric i.e. $g(x_p|x)=g(x|x_p)$? I'm not sure whether it ...
26 views

82 views

### Modified gambler's ruin problem: quit when going bankruptcy or losing $k$ dollars in all

In each round, the gambler either wins and earns 1 dollar, or loses 1 dollar. The winning probability in each round is $p<1/2$. The gambler initially has $a$ dollars. He quits the game when he has ...
57 views

12 views

30 views

### What is the difference between a reversible markov chain and a reversible in equilibrium markov chain?

In the text I'm using it says: Let X = {$X_n : 0 \leq n \leq N$} be an irreducible Markov chain such that $X_n$ has the stationary distribution $\pi$ for all $n$. The chain is called reversible if ...
18 views

### stopping criteria for power-iteration to find rank-1 matrix

I start with B=I, A positive matrix, and compute B=(BA)/norm(B) by iterating until B is sufficiently close to rank-1 matrix. What is a good stopping criterion for this algorithm? There's Birkhoff ...
26 views

### Show that it is a Markov chain, determine the transition-probability matrix and reversibility

Certain machine has three possible states: $0=working,\,1=broken\,and\, awaiting\,repair,\,2=broken\,and\,being\,repaired$. The permanence times (in minutes) in each state have independent geometric ...
34 views

### Distribution of throws of die rigged to never produce twice in a row the same result

A die is “fixed” so that each time it is rolled the score cannot be the same as the preceding score, all other scores having probablity 1/5. If the first score is 6, what is the probability that the ...
19 views

### Random walk mean number of visits to state before absorption

This is from Stirzaker's book Random Processes. Suppose we have a simple random walk with probability going "up" p, "down" q. At time 0 it stats at 0, so $$S_0 = 0$$ Now let $u_b$ be the mean ...
### On the Markov chain defined by $X_n=U_nU_{n+1}$, where $(U_n)$ is i.i.d. symmetric Bernoulli
I came across this problem in homework: $U_n$ are i.i.d random variables with $P[Un=1]=P[Un=−1]=0.5$. a) Show that $X_n=U_nU_{n+1}$ is a Markov Chain. b) Show that $X_n=(U_n+U_{n+1})/2$ is not a ...