Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

795 views

When the sum of independent Markov chains is a Markov chain?

I try to find as much as possible cases, when the chain $Z(t) = |X_1(t)-X_2(t)|$ is Markov, where $X_1(t)$ and $X_2(t)$ are independent, discrete-time and space, preferably non-homogeneous Markov ...
181 views

recurrence criterion for random-walk like Markov chain

Suppose we have a random-walk like Markov chain, i.e. state space is the set of all integers from $-\infty$ to $\infty$, the transition probability $P_{ij}$ is nonzero only when $j=i+1$ or $j=i-1$. ...
133 views

Log-likelihood function

I'm not sure if this could be asked here, or in math overflow... In the following paper Cho, Jin Seo, and Halbert White. "Testing for regime switching." Econometrica 75.6 (2007): 1671-1720. doi: ...
1k views

Strong Markov property - Durrett

I recently had great success with my first question here so I will boldly go on to a second. Here goes: I'm studying Markov Chains in Rick Durrett - Probability: Theory and example and I'm stuck with ...
502 views

155 views

846 views

Simple proof that stationary birth-death chains are reversible

A Markov chain with state space $\mathbb{Z}$ is a birth-death chain if the transition probabilities satisfy $p(x,y) = 0$ for $|x-y| > 1$. That is, the only possible transitions are to move one ...
2k views

Invariant Probability Vector

I'm reading through my textbook, Introduction to Stochastic Processes (Lawler), before the semester begins in hopes of getting ahead, and I've run into something I just plain cannot figure out: How to ...
696 views

How can a Markov chain be written as a measure-preserving dynamic system

From http://masi.cscs.lsa.umich.edu/~crshalizi/notabene/ergodic-theory.html irreducible Markov chains with finite state spaces are ergodic processes, since they have a unique invariant ...
813 views

Tricky Probability question

Each morning a student takes one of the three books he owns from his shelf. The probability that he chooses book $i$ is $a_i$, where $0 < a_i < 1$ for $i=1,2,3$ and the choices on successive ...
9k views

Expected number of steps/probability in a Markov Chain?

Can anyone give an example of a Markov Chain and how to calculate the expected number of steps to reach a particular state? Or the probability of reaching a particular state after T transitions? I ...
156 views

Probability of finding 2012 before any other occurence of 012 in a random infinite sequence of digits 0,1,2

The following problem is from the semifinals of the Federation Francaise des Jeux Mathematiques: One draws randomly an infinite sequence with digits 0, 1 or 2. Afterwards, one reads it in the ...
500 views

Are primitive row stochastic matrices diagonalizable?

Let $A$ be an $n \times n$ matrix with real, non-negative entries. Assume $A$ is primitive, meaning there exists an integer $k$ such that $A^k>0$ (here the inequality means all entries in $A$ are ...
6k views

How can I compare two matrices?

I have a matrice A. It is model probability matrice for some process (Markov chain). Then, I have estimated matrice B. I have to somehow compare these two matrices to tell whether process that gave ...
51 views

What is the expected number of steps in a random walk from leaf to leaf in a full binary tree?

Let $h \geq 2$ be a natural number. Consider a complete binary tree of height $h$. Say we take a random walk starting from the "leftmost" leaf. What is the expected number of steps before the "...
104 views

I'm interested in all possible paths (on the grid $\mathbb{N}^2$) that goes from $(0,0)$ to $(n, n)$. At each step there are two possibilities: go right or go up. The path is a sequence $z=(z_0,... 1answer 200 views A die whose score cannot be as before (Markov chains) A die is "fixed" so that each time it is rolled the score cannot be the same as the preceding score, all other scores having probability$1/5$. Given that the first score is 6, what is the probability ... 2answers 145 views When are stable continuous time Markov chains Feller? Always? This is a question is similar to this 2 year-old one that never got answered (truthfully it's pretty much the same question except that I'm adding a bit more detail and the assumption that the$Q$... 2answers 127 views Markov and independent random variables This is a part of an exercise in Durrett's probability book. Consider the Markov chain on$\{1,2,\cdots,N\}$with$p_{ij}=1/(i-1)$when$j<i, p_{11}=1$and$p_{ij}=0$otherwise. Suppose that we ... 1answer 49 views Stopping time in Markov chains A random variable$T : \Omega \rightarrow ${$1,2,3...$}$\cup${$ \infty$} is called a stopping time if the event {$T=n$} depends only on$X_0 , X_1 ,X_2 ,..., X_n$for$n = 0,1,2,...$I have trouble ... 1answer 120 views Doubly stochastic matrix proof A transition matrix$P$is said to be doubly stochastic if the sum over each column equals one, that is$\sum_i P_{ij}=1\space\forall i$. If such a chain is irreducible and aperiodic and consists ... 1answer 281 views Find the Stationary Distribution of an infinite state Markov chain A Markov Chain on states 0,1,..... has transition probabilities$P_{ij}=1/(i+2)$for j=0,1,....,i,i+1. I'm supposed to find the stationary distribution. So do I take the limit as n goes to ... 2answers 159 views Expected value of money left from a coin flipping game Say we were to play a game. We started off with \$100 and kept flipping a fair coin. If it turned out heads, we won \$1, else our money got inverted. For example, if on the first flip we got heads, ... 1answer 98 views random walk in a certain environment Consider the following random walk in one dimension, starting from$r(0)=0$. $$r(i+1) = r(i) + \xi,$$ where$\xi(i, r(i))$is an increment with distribution$P(\xi=1) = \frac{c^{r(i)}}{i-r(i)+1}$... 2answers 728 views Random walk on lollipop graph Hi i am trying to prove expected Hitting time on the Lollipop graph. It is a graph on$n$vertices with clique on$n/2$vertices and path joined to this. Let vertex$i$be a vertex on the clique, ... 1answer 1k views Return time of a markov chain I'm having trouble deriving the return time for a Markov chain. The graph has$n$vertices and is connected by$n - 1$edges. So we can draw this as a horizontal line of nodes with node$1$all the ... 1answer 174 views Chance of being able to quit while ahead in a betting game (Markov chain with gambler's ruin) Suppose a player starts with$N$chips, and is playing a game with odds$O$, betting 1 chip in each iteration. When the player reaches 0 chips the betting must end. What is the probability that at ... 1answer 196 views Combinatory + Coding Theory I am reading about an algorithm for ﬁnding minimum-weight words in large linear codes. Let$c$be the codeword of weight$w$to recover (with size$n$and in$GF(2)$). Let$N = \left\{1, 2, \ldots, n\...
For discrete time finite Markov chain, the first passage time $T_j$ to visit state $j$, is determined from the recurrence equation:  p^{(n)}_{ij} = \sum_{k=0}^n f_{ij}^{(k)} p^{(n-k)}_{jj} = ...