Skip to main content

New answers tagged

0 votes

Random walk on the edges of a square where staying at same position is allowed

Let $S$ be the total number of step and R be the number of real steps. Then the probability for i steps is: $$\mathbb{P}(S=i)=\sum_{n=1}^{\lfloor i/2\rfloor}\mathbb{P}(S=i|R=2n)= \sum_{n=1}^{\lfloor i/...
Henkie's user avatar
  • 106
1 vote

Random walk on the edges of a square where staying at same position is allowed

You are right that the new answer is $6$. Before every "useful" step that the walk makes, it makes an average of $0.5$ "fake" steps (i.e. steps where it stays at the same site). We ...
James Martin's user avatar
0 votes

Markov Chain Detailed Balance $\pi(x)*P(x, y) = \pi(y)*P(y, x)$

For an irreducible $P$ such that $P(x,y)>0$ if and only if $P(y,x)>0$ (all pairs of states $x$ and $y$), detailed balance hold for some reversibility measure $\pi$ if and only if Kolmogorov's ...
John Dawkins's user avatar
  • 26.5k
3 votes

What are the stationary distributions of this Markov Chain?

There are two closed irreducible classes of this Markov chain, namely $C_1=\{1,2,3\}$ and $C_2=\{7,8\}$. You need to find the unique stationary distributions corresponding to $C_1$ and $C_2$. Call ...
StubbornAtom's user avatar
  • 17.3k
2 votes

Time-dependent transition probabilities

Here is an alternate proof that doesn't use the hint. Label the cigars in order that they get selected for the first time. Let $E_i$ be the event where the $i$th cigar gets selected a second time ...
mercio's user avatar
  • 50.4k
2 votes
Accepted

Time-dependent transition probabilities

Your recurrence relation is correct (apart from a small misprint) and being supplemented with the boundary condition $v_n(0)=n$ is in fact solved by the expression given as a hint. Your expression: $$ ...
user's user avatar
  • 26.7k
2 votes
Accepted

For which values of $a$ is this Markov Chain irreducible and aperiodic?

For the first bit, yes, you are essentially right. To prove this for small chains, I like to enumerate the states (say, as $1,2,3,4,5$), then (assuming you start in the $i$th row and move to the $j$th ...
PrincessEev's user avatar
  • 45.1k
2 votes

Decomposing a general stopping time into stopping components

(The $y$ in Frid's notation is misleading.) Instead, for a stopping time $\tau\ge 1$ and a state $x$, define $T_x(\omega):=\tau(x\omega)-1$. (Here, for a path $\omega=a_0a_1a_2\cdots$, the ...
John Dawkins's user avatar
  • 26.5k
1 vote
Accepted

Are Markov chains $X_{t+r}$, $X_{2t}$ and $(X_t; X_{t+1})$?

Since $X_k$ is a Markov chain we have $$ \mathbb{P}(X_{2n+1}=k\mid X_{2n}=i_0, X_{2n-2}=i_2,\dots) =\mathbb{P}(X_{2n+1}=k\mid X_{2n}=i_0) $$ Next, $$ \mathbb{P}(X_{2n+2}=i\mid X_{2n}=i_0, X_{2n-2}=i_2,...
van der Wolf's user avatar
  • 3,367
1 vote
Accepted

Coupling identical Continuous-Time Markov Chains

First, I assume the state space, say $E$, is countable since you are using indices $i,j$ to denote arbitrary states. This assumption is crucial for our discussion. 1. Positive Recurrence An ...
Hirofumi Shiba's user avatar
1 vote
Accepted

Finding References for known results about mixing time of a Markov chain arising from Gaussian elimination

There are at least $N=c2^{n^2}$ invertible matrices mod 2, where $c=1/2(1-1/4)(1-1/8)\cdots (1-2^{-k})\cdots>0$. Since there are less than $n^2$ possible moves in each step, after $t$ steps the ...
Yuval Peres's user avatar
  • 22.2k
2 votes

Proof of Chapman-Kolmogorov equations for the general case

Let $x_t$ be the state $t \geq 0$ period ahead \begin{align*} Q^{n+m}(x,\,A)& = \int Q^{n+m-1}(x_1,\,A)\,Q(x,\,d\!\,x_1)\\ & = \int \Bigl(\int Q^{n+m-2}(x_{2},\,A)\,Q(x_1,\,...
chenyongtxt's user avatar
6 votes
Accepted

circular random walk - markovian frog

The set of states the frog has visited is always a contiguous block. It can grow on both ends as the frog can traverse back and forth in the visited block. That block needs to grow $n-1$ times. The ...
ploosu2's user avatar
  • 9,548
0 votes

Prove that uniform distribution on a set of vertices $V$ is stationary if the graph is regular.

Let $G=(V,E)$ denote a regular graph. Let $|V|=n$. The stationary distribution is given by $\pi(u) = \frac{\text{deg}(u)}{2|E|}$ (see for why). Let $d=\text{deg}(u)$ for all $u \in V$. Then $|E| = \...
Ari's user avatar
  • 101
1 vote

When do these Algebra Equations have solutions?

It seems this is a studied topic. But since I am a specialist in algebra, but not in the probability theory or Markov chains, I studied my source and to follow it. This is [Gan, Chapter XIII, $\S$6-7.]...
Alex Ravsky's user avatar
  • 92.7k
1 vote

Discrete Hidden Markov Model with continuous observation

Hidden Markov models with continuous observation variables are very common in some fields (e.g., finance, ecology). All the common algorithms can directly be extended to the continuous case by ...
Théo Michelot's user avatar

Top 50 recent answers are included