New answers tagged

0

This is how I would view the symmetry associated with the stationary distribution: Put each ball in either urn with equal probability. Pick that ball up a random number of times unrelated to which urn it is in, and move it to the other urn. The probability that that ball started in urn A was $\frac12$ and in urn B also $\frac12$. Similarly with its ...


0

The Markov property extends to stopping times. Just as we have $\sigma$-fields $\mathscr{F_n}$ associated with constant times, we do have a $\sigma$-fields $\mathscr{F_{\tau}}$ associated to any stopping time. This $\mathscr{F_{\tau}}$ is the information we have when we observe the chain up to time $\tau$. One can check from the definition that $\tau$ ...


0

Since all entries of $A^2$ are strictly positive, so are all entries of $A^n$ for all $n > 2$. The period of index $4$ is the gcd of all $n$ such that $(A^n)_{44} > 0$, so that is $1$, not $2$.


1

The other answer is incorrect in saying that you need detailed balance. Intuitively, the equation in the OP says that the probability going into state $i$ is equal to the probability going out of state $i$, when the distribution is the stationary distribution. But the difference between these two quantities is $(\pi P - \pi)_i$, which is zero by the ...


0

I'm not sure how accurate a result you require, but regardless of the lattice, discrete time Markov chain embedded in $|X_t|$ is a biased random walk that gets absorbed at $0$. From this, you can easily compute the limit as $t\to\infty$ of the survival probability. The papers by Bramson and Griffith on the "Williams-Bjerknes tumor growth" model give bounds ...


1

For brevity, I'm going to say "we win" if the game stops with a sum of $5$, and "we lose" if the game stops with a sum of $7$. Let $p_k$ be the probability that we win, given that the first roll is $k$. Suppose we roll a $1$ and then a $2$. After the $2$, the probability that we will win is $p_2$; that is, all that matters after that roll is that we last ...


1

I think you need the condition that the Markov Chain is reversible. Def: Let $X$ be an irreducible Markov chain such that $X_n$ has the stationary distribution $\mathbb{\pi}$ for all $n$. The chain is called reversible if the transition matrices of $X$ and its time-reversal $Y$ are the same, which is to say that $$\pi_iPij = \pi_jPji \:\:\:\: \forall ...


1

I'd consider $8$ states, namely for $1\leq k\leq 6$ the states $s_k:\ $"last roll was $k\>$, but game is not yet over", and the two end states $e_5$ and $e_7$. Denote by $x(n)$ the $(1\times8)$ row vector giving the probabilities that after $n$ rolls we are in the state $s_1$, $s_2$, $\ldots\ $, $s_6$, $e_5$, $e_7$ respectively. It follows that ...


1

It is not quite true that any random walk on $\mathbb{Z}_n$ is irreducible. This will depend on the increment distribution $\mu$ as well. For example, on $\mathbb{Z}_4$ if you take $\mu(g)=\delta_{g+2}$ then there are two communicating classes $\{0,2\}$ and $\{1,3\}.$ There are lots of different stationary distributions for this random walk.


3

If the chain has an absorbing state, then the answer is no, because then $P(X_0=x_0,...X_n=x_n)>0$ but $P(X_n=x_n,...,X_0=x_0)=0$, where $x_n$ is the absorbing state. On the other hand, suppose your markov chain can be broken down into two disjoint communicating classes, i.e. two disconnected Markov chains. Then if the individual markov chains are ...


0

I see you already got an answer. I thought I could add some details if you are curious. The state 4 is required for Q to be tied to a probability measure. Without it any previous 3-in-a-row would "leak out" and the total probability would no longer sum up to 1. Matrices like Q must always have an eigenvalue equal to $1$ for this to be true. If we calculate ...


1

The sum of each column in transition matrix has to equal 1 (consider a perfectly located state in the beginning - after the matrix multiplication the probability vector will represent the corresponding matrix column). In your case it would be equal to zero. You can also think of it this way - you would converge to a state where the imaginary particle is ...


1

First of all, notice that $Z_{n+1} = 2 \Leftrightarrow X_{n+1} = 2$ and $Z_n = 0 \Leftrightarrow X_n \in \{0,1\}$. So $$P(Z_{n+1} = 2 \mid Z_n = 0, Z_{n-1} = 2) = P(X_{n+1} = 2 \mid X_n \in \{0,1\}, X_{n-1} = 2).$$ Write this quantity as $$P(X_{n+1} = 2 \mid \{X_n = 0\} \cup \{X_n = 1\}, X_{n-1} = 2).$$ The strong Markov property says that you can drop the ...


2

$\Bbb P[X_0=x_0,X_1=x_1]=\pi(x_0)P(x_0,x_1)$ and $\Bbb P[X_0=x_1,X_1=x_0]=\pi(x_1)P(x_1,x_0)$, so the $n=1$ case of reversibility already implies detailed balance.


2

You are right to be skeptical as there are some conditions missing. For example, $\mathbb{P}(T_{s_1}<T_{s_2}\mid X_1=s_1)=1$, but there is no guarantee that $b_{s_1}=1$. The equation $b_i=\sum_{j\in S}b_j\, p_{ij}$ will be true if the initial state $i$ is so "far away" from $s_1$ that $X_1=s_1$ is impossible.


1

As far as I can see, $p$ is just a distribution of a i.i.d random variable $X_i$ taking values on $\mathbb{Z}^d$. With respect to chosen generating set, any member of $\mathbb{Z}^d$ is represented as a coordinate vector $(k_1,\cdots,k_l)$ (using notation used in the original article. Now $p$ is saying that $P(X_i=(0,0,\cdots,0,\pm 1,0,\cdots,0)) = ...


1

It is implied (and it should have been said) that all other probabilities are zero, i.e., the probability is supported by the origin, the basis vectors and their opposites; $p(y)=0$ unless $y$ is one of the $x_k$ or its opposite or the origin. The interpretation is that a random walk has probability zero of taking two jumps (including diagonal jumps) at a ...


0

An approach that may be along the lines of what you suggest you need to use is the following: If you denote the probability $P \{\text{ETS C} | X_0 = i\}$ by $q_{iC}$ , where ETS is "eventually the state", then using the Law of Total Probability gives $ q_{iC} = \sum_{j \in S} P \{\text{ETS C}, X_1 = j | X_0 = i\}$ and using the definition of conditional ...


0

As I wrote above, there are eight states, which can be represented in this graph: The numbers in each vertex describe the number of heads, then tails and whether the heads or the tails should be counted for the next transition. (Be sure the transitions away from any vertex sum to 1.0.) It is a simple matter to label each edge by the probabilities of ...


0

Let $H_n\in \{0,1,2,3\}$ be the number of heads after iteration $n$. Obviously, $H_n+T_n=3$ Then $H_n$ would be enough to denote the state (we'd have then 4 states) - but then the Markov Chain would not be stationary (the transitions would depend on time). We want to avoid that, normally, if it's possible. Let's add then an indicator variable $W_n=\{0,1\}$, ...


1

In order to find the steady-state vector $s = \begin{pmatrix} s_1 \\ s_2 \end{pmatrix}$ you need to solve a simple matrix equation \begin{equation} (T - I)s = 0. \end{equation} But \begin{equation} T - I = \begin{pmatrix} -\frac{1}{2} & 1 \\ \frac{1}{2} & -1 \end{pmatrix} \end{equation} and, thus, you have a linear system \begin{equation} ...


1

To see a counterexample to your argument, let for example the transition probabilities: $6 \to 5= 0.01$ and $6 \to \{3,4\}=0.99$, $5 \to 6= 0.01$ and $5 \to \{1,2\}=0.99$. Now, a simple calculation shows that $$P_6[T_{1,2}<+\infty]\le 0.01 < 0.99 \le P_5(T_{1,2}<+\infty)$$ For them to be equal, you need certain symmetry conditions on the ...


0

The Markov property implies that the process has no memory outside of its current state. This means that return probabilities are mutually independent in the sense that the probability of returning from a state $i$ to itself $n$ times is the $n$-th power of individual return probabilities. Consider the probability $R(i)$ for, given initial state $i,$ ...


1

maybe this helps. page 153 ff. https://www.math.ucdavis.edu/~gravner/MAT135B/materials/ch13.pdf You will also find the definitions for the transition rates there by summing over probabilities P[X_n=j|X_0=i] and the expected mean time of recurrence. So if some probabilities P[X_n=j|x0=i] of returning back to i in n steps are <1, by definition you ...


0

Firstly observe that \begin{align}\left(X_{n+1}-X_n \mid X_n=i\right)&=\left(\left( i+Z_{n+1}\right)_+-i\right)\\[0.2cm]&=\begin{cases}i+Z_{n+1}-i,& \text{if }Z_{n+1}+i\ge 0\\0-i,& \text{if }Z_{n+1}+i< 0\end{cases}\\[0.3cm]&=\begin{cases}Z_{n+1},& \text{if }Z_{n+1}\ge -i\\-i,& \text{if }Z_{n+1}<-i ...


2

$$X_{n+1}-X_n = ((X_n+Z_{n+1})\vee 0)-X_n=Z_{n+1}\vee (-X_n) \Rightarrow$$ $$\mathbb{E}[X_{n+1}-X_n\mid X_n=i]=\mathbb{E}[Z_{n+1}\vee (-i)]=\mathbb{E}[Z_1\vee (-i)]\to \mathbb{E}Z_1<0$$ as $i\to \infty$ (assuming that $Z_1\in L^1$).


1

The generator matrix is: $$ G= \begin{bmatrix} -2 & 2 & 0 & 0 \\ 3 & -5 & 2 & 0 \\ 0 & 3 & -5 & 2 \\ 0 & 0 & 3 & -3 \\ \end{bmatrix} $$ The balance equations in vector form are: $\mathbf{\pi G = 0}$, so, working down each of the four columns in turn: \begin{align} ...


0

The double integral is nested, and you cannot "separate" them. Since $t_1 < t_2 < t_3 < t_4$, to obtain the joint pdf of $(t_1, t_4)$, we can integrate by $$ \int_{t_1}^{t_4}\int_{t_2}^{t_4} f(t_1, t_2, t_3, t_4) dt_3dt_2 = \int_{t_1}^{t_4}\int_{t_1}^{t_3} f(t_1, t_2, t_3, t_4) dt_2dt_3 $$ So the limits depends on the order of integration. If you ...


2

The solution in image. I will find some time to type in latex later. The solution uses the result of a previous problem.


0

Defining a harmonic function $u$ in terms of its spherical averages, instead of the Laplacian, gives a better analogue to the discrete averaging property $h(x)=\sum_{y\in\Omega} P(x,y)h(y)$ of the function $h$.


2

Suppose you have a uniform grid of points in the domain $\Omega$. Each of these points has a certain number $N(x)$ of neighbors (most of them have $2n$ of them). If you introduce a Markov chain with $P(x,y)=1/N(x)$ if $x$ and $y$ are neighbors and $0$ otherwise, then a harmonic function $h$ in the sense of your probability definition is a finite difference ...


1

Say your generator matrix is: $$ G = \begin{bmatrix} -\lambda_0 & \lambda_{01} & \lambda_{02} & \cdots \\ \lambda_{10} & -\lambda_{1} & \lambda_{12} & \cdots \\ \lambda_{20} & \lambda_{21} & -\lambda_{2} & \cdots \\ \cdots \\ \end{bmatrix} $$ where, as usual, for each ...


0

First, there is no such thing as “the” eigenvector, since any non-zero scalar multiple of an eigenvector is also an eigenvector for the same eigenvalue. Clearly, the eigenvector of $1$ that you found and the one the author uses are scalar multiples of each other, so they are equivalent. The author has normalized it so that the sum of its components is ...


0

The rates for going from an absorbing state $i$ to another state $j$, $q_{ji}$ with $i\neq j$, are zero. Then the Kolmogorov's criterion is verified if $i$ is included in the loop, since you have zeros at both sides (the loop is closed, so whenever $i$ is included, both $q_{ij}\geq0$ and $q_{ji}=0$ have to be present). So, in principle, the CTMC could be ...


2

For $kr\le t\le (k+1)r$, $P(\tau > t)\le P(\tau>kr)$, so$\sum_{t\ge0}P(\tau>t)=\sum_{k\ge0}\sum_{i=0}^{r-1}P(\tau>kr+i)\le\sum_{k\ge0}rP(\tau>kr).$


1

Once you have got 3 consecutive heads, your goal is attained, and the task is over. 1.The probabilities in question obtained after N trials are the cumulative probabilities. 2. No, you can't say that. $P(N \le 8)$ is the probability that at most $8$ flips are needed. you can understand from the probability values for trials 3 through 8 $3\quad 0.125 000 ...


1

This is the very first application of the first step analysis, which introduce the usage of law of total expectation and the Markov property. Let $X_{t}$ and $X_{t+1}$ be the current state and next state respectively, $N_{t}$ be the number of throws to reach the absorption state $3$ at time $t$. Then for $i \neq 3$, $$ \begin{align*} \phi_i & = ...


0

What are the possible values of $X_m$? For a certain value $x$ of $X_m$, what are the possible values of $X_{m+1}$? (Hint: you could pick two blacks, a red and a black, a black and a red, or two reds.) If $X_m$ is $x$, what is the probability that $X_{m+1}$ is $y$? (Hint: this depends on the probability of picking two blacks, red and black, black and red, ...


0

To see why for $p \neq 1/3$ the process is transient, just apply Law of Large Numbers. Let $X_i \in \{ -1, 2 \}$ where $X_i = -1 $(with probability $1-p$) and $X_i = 2$ (with probability $p$). Then, we are talking about the process $(S_n)_n$ where $S_n = \sum_{i=1}^n X_i $, right? Well, we have $\mathbb{E}[X_i] = 3p-1$, thus, using independence of $X_i$'s, ...


0

0 and 5 are absorbing states. Once entered, the process stays there. For states 1,2,3,4, there is some probability you will immediately win enough bets in a row to get to 5 or lose enough to get to 0 (the absorbing states). Since, for which ever one of these states you start at, there is a non-zero probability of never returning, the states are transient. ...


2

Here, state 4 is needed to sort out the difference between "the third heads just happened" and "the third head happened before now". The probability that it takes 8 flips to get 3 heads in a row is the probability the process is in state 3 just after the 8th flip (that is, the third heads happened on the 8th flip). Notice, if the state is 4 just after the ...


1

Conditioned on any state, the progression of a Markov chain is independent of any earlier state, so yes, $$ P(X_{n+m} = j \mid X_n = k, X_0 = i) = P(X_{n+m} = j \mid X_n = k) $$ and that means also that you are right, that $$ P(X_{n+m} = j \mid X_{n+1} = b, X_n = k, X_0 = i) = P(X_{n+m} = j \mid X_{n+1} = b) $$



Top 50 recent answers are included