# Tag Info

1

I believe the best strategy for a problem of this kind would be to proceed in two steps: Fit a continuous time Markov chain model to the data by estimating the (infinitesimal) generator $Q$. Using the estimated generator and the Kolmogorov backward equations, find the probability that a Markov chain following the fitted model transitions from state $i$ to ...

1

It is indeed quite confusing, also taking into account that we often write $$\int P(x,A)\mu(\mathrm dx)$$ even though some may prefer $\int \mu(\mathrm dx)P(x,A)$ - in the latter case I'm often confused where does the integral end. Nevertheless, it seems that the notation for measures and kernels comes from the fact that functions over finite spaces are ...

0

Choose $p_{2x}=\frac12\left(1-a\right)$ and $p_{2x-1}=\frac12\left(1+b\right)$ for every $x\geqslant1$, with $a$ and $b$ in $(0,1)$. For every $x\geqslant1$, the chain starting from $2x$ hits $2x+2$ before $2x-2$ with probability $$p=\frac12\left(1+\frac{b-a}{1-ab}\right).$$ Thus, the chain is transient if $p\gt\frac12$ and (positive) recurrent if ...

2

Let $p_{jk}=2k/j^2$ be the one step transition probability. Then from state $m+1$ we can only get to state $m$ if we jump there in the next step, so $W_{m+1,m}=p_{m+1,m}=2m/(m+1)^2.$ Note that this expression satisfies the general formula for $W$ so we can start the induction here and continue to $W_{m+2,m}, W_{m+3,m},...$ Starting from state $j$ we ...

0

Yes, $(1,3)$ is the probability of landing on 3 having started at 1. And generally, if you're interested just in the probability of landing on a particular space at any point in the game, the first row is indeed all you need.

0

If your desired left eigenvector is a row vector $x$ with $x_i\ge0$ and $\sum x_i =1$, then the matrix $$P= \begin{pmatrix} x \\ x\\ \vdots \\ x\end{pmatrix}$$ has $x$ as left eigenvector to the eigenvalue $1$.

1

Those are two equivalent descriptions of the same semi-group. You mentioned how to deduce $q$ from $(p_t)$, note that, conversely, $p_t=\mathrm e^{tq}$ for every $t\geqslant0$.

1

Let $R_j$ denote the first return time to $j$, then (strong) Markov property at time $R_j$ yields$$P(X_n=j\mid X_0=j)=\sum_{m=1}^nP(R_j=m\mid X_0=j)P(X_{n-m}=j\mid X_0=j).$$ Nota: It might be time for you to get a good textbook since the proof above is in all those on the subject. You might try Markov chains by James Norris, available on the web.

1

Consider a trajectory of $n$ transitions, ending in state $j$. Either transition $n$ is the first return to state $j$, which has by definition has probability $f_j^{(n)} = f_j^{(n)} p_{jj}^{(0)}$, or there was a return at some earlier transition(s), the first of which we can label transition $m$. If first return was at $m \neq n$ (which itself has ...

0

One can decompose the final formula, this shows how the proof works. Assume wlog that $m\gt0$. $\frac12=P(S_1=1)$ (if $S_1=-1$, then $V_m=0$) $\frac1m=P_1(T_m\lt T_0)$ (means that one gets at least one visit to $m$) $\frac1{2m}=P_m(T_0\lt T_m)$ (means that the number of visits does not increase anymore) $1-\frac1{2m}=P(V_m\geqslant k+1\mid V_m\geqslant k)$ ...

1

Is there any known algebraic relation between these two matrices? No nonambiguous one, and for a good reason: there is much more information in the latter than in the former.

1

Here's is an answer using a slightly different approach. We denote the $n$ players with $player_1$ to $player_n$ according to the order of their duels. So the first duel is between $player_1$ and $player_2$, the next duel is the winner of the first duel with $player_3$ until finally $player_n$ has his chance to win the game. Let's denote with $p_k$ the ...

1

As often with physicists' notations, everything works fine if one knows what is going on, but otherwise... Here one considers a jump process $(\sigma_t)_{t\geqslant0}$, and, for every $t$, the random variable $$W_t=\sum_{0\lt s\lt t}\omega(\sigma_{s^-},\sigma_s)\,\mathbf 1_{\sigma_{s^-}\ne\sigma_s},$$ where, for every positive $s$, ...

0

If the derivation looks anything like the proof of the Crooks fluctuation theorem, which it appears to resemble at first sight, then the expected value $\mathbb{E}_\sigma$ is carried out over all possible trajectories starting at $\sigma$, and not all possible states, which is what the angle brackets denote. The quantity $\omega(\sigma,\sigma')$ represents a ...

1

Answer: It does not follow. I don't know what your lecturer meant or said, but you cannot draw this conclusion, as Michael's example shows. Let $(X_n)$ be a Markov chain with state space $\{0,1\}$ and transition matrix $P=\pmatrix{0&1\cr 1&0}$. The unique invariant distribution is $\pi=(1/2,1/2)$, but $\mathbb{P}_0(X_n=0)=(-1)^n$ does not converge ...

0

You began with a set $A$ with $\phi(A)>0$, and deduced that $$K^n(x,A)\geq {1\over m} \, 1_B(x)\tag1$$ for some $m$, $n$, and $B$ with $\pi(B)>0$. Now integrate both sides of (1) with respect to $\pi$ to obtain $$\pi(A)=\pi K^n(A)\geq {1\over m} \, \pi(B)>0.$$ Since $\phi(A)>0$ implies $\pi(A)>0$, you've shown that $\phi\ll \pi$.

0

Rule 1 doesn't affect the process much: Given a standard random walk of the form you initially describe, recursively removing backtracks (e.g. the subsequence ULRRLD would get removed) yields a walk following rule 1, and importantly, does not introduce any bias into the process (i.e. the three available options (under rule 1) are equally likely also in this ...

1

$Y_n$ is not a Markov Chain, $Z_n$ is a Markov Chain. Your proof is clearly wrong. You can easily find an example where the Markov Chain is not a random walk Even take the Markov Chain with two states: 1<->2 where $p_{12}=0.3=p_{21}$, $0.7=p_{11}=p_{22}$. Basically if you only know the sum up to time $n$, assuming that $Y_n\geq 2$, you don't know which ...

1

The probability the Markov chain ends up in one of the absorbing states is $1$. Let $p_i$ be the probability of absorption in state 1 when starting in state $i$ Note $p_2=0$ $p_1=1$ $$p_3 = 0.2p_1+0.6p_3+0.2p_4$$ why?! write down the equation for $p_4$, then solve. Let $q_i$ be the probability absorption in state 2, note that $q_i+p_i=1$ (why?)

1

You can also simply use the time-reversibility equation since it is a birth-death process. These means the rate into state $i$ from $j$ must equal the rate out of state $j$ into state $i$. More formally, $$\pi_i q_{ij} = \pi_j q_{ji}$$ So, our system of equations are: \begin{align} \lambda \pi_0 &= \mu \pi_1 \\ \tfrac{\lambda}{2}\pi_1 &= \mu ... 2 By the way, there is a typo in the post. The recurrence relationship is meant to saya_{k+1}-a_k = \frac{p_k}{q_k}(a_k-a_{k-1})$$Let's take u_k= a_k-a_{k-1},$$u_{k+1}= \frac{p_k}{q_k}u_k\tag 1$$using the fact a_0=0 and a_1=1, we have$$u_1+u_2+\cdots+u_n=1\tag 2$$Repeated applying the recurrence relation (1), we see that for 2\leq i\leq ... 1$$\pi(3)=\frac{\lambda^3}{\lambda^3+6\mu\lambda^2+12\mu^2\lambda+12\mu^3}$$1 "Is the inequality necessary at all here (n<N)?" Yes, since you have N states, then the longest way (if it exists) will be N-1. If the chain is finite and if there is a integer such that p^n_{ij}>0 can't we say that j can be reached by i without considering the number of states? Yes. The main problems here are absorbing states ... 3 Let t_i denote the mean time to absorption at state 0 starting from state i then t_0=0, one is after t_2, and the usual one-step Markov property yields$$t_2=1+\frac15t_1,\qquad t_1=1+\frac7{10}t_1+\frac15t_2,$$which is solved by$$ t_2=\frac{25}{13}=1.923\ldots For n+1 states, one gets an n\times n affine system whose solution (t_i) ... 3 We conventionally read row by row, but it appears that in your matrix, the columns add to 1 instead. I'll write your matrix like this and proceed: \begin{align*} P = \begin{bmatrix} 0 & 0.2 & 0.8 \\ 0.2 & 0.7 & 0.1 \\ 0 & 0 & 1 \end{bmatrix} \end{align*} which is of the formP=\left[ \begin{array}{c|c} Q & R \\ ...

0

One can forget Server 2. The length of the queue at Server 1 increases by $1$ at rate $a$ and, when positive, it decreases by one at rate $u_1$. This is the most classical birth-and-death chain, whose stationary distribution exists if and only if $a\lt u_1$ and can be readily found, either reading your textbook or solving the associated system of linear ...

0

(c) is a Markov chain, since the transition at step $n+1$ only depends on the value of the sum up to $n$ (the endpoint of the random walk) and the maximum thus far. (b) is not a markov chain unless $p=\frac{1}{2}$ and a counter-example is to take $n=1$; then $|S_n| = 1$ but $P(|S_2|=2) = p$ if the first step was to $-1$ but is $1-p$ if the first step was to ...

Top 50 recent answers are included