# Tag Info

Accepted

### Prove that $P(X_{n+m}=j|X_0=i_0,\dots, X_n=i)=p_{ij}^{(m)}$

I will sketch the proof of the case $m=2$; the general case is analogous. Let $E_{n,i} = \{X_0=i_0, \ldots, X_n=i\}$. \begin{align} P(X_{n+2} = j \mid E_n) &= \sum_{k} P(X_{n+2} = j, X_{n+1} = k \...
• 90k
1 vote

### A question about Markov chain's definition

It is not necessary: if you know that $$P(Z_{n+1}=y \mid Z_0=x_0,\dots,Z_{n-1}=x_{n-1},Z_n=x)=P(Z_1=y \mid Z_0=x)$$ then you can show that both of those are equal to $P(Z_{n+1}=y\mid Z_n = x)$. To ...
• 143k

### Exercise on irreducible positive recurrent markov chain

1. Let $N_n(A)$ be the number of visits to $A$ by time $n$; that is, $N_n=\sum_{j=1}^n 1_A(X_j)$. What can you say about $N_n(A)/n$ as $n\to\infty$? 2. Observe that, for each $n$ and $k$, $T_k^A\le n$...
• 26.1k

### Markov process characterization and application of monotone class theorem

It's my first time answering questions in stochastic processes, so please correct me if I'm wrong. Following @TheBridge's comment, we can use the Functional Monotone Class Lemma (Theorem 2 in ...
1 vote

### Properties of a transient state in a Markov Chain

As you have pointed out, $\sum\limits_{n=1}^{\infty}P_{jj}^{n}<\infty$, and that $\mathbb{P}_{ij}(s)=\mathbb{F}_{ij}(s)\mathbb{P}_{jj}(s)$, what you need follows directly from Abel's theorem, since ...
Accepted

### Why does the Wiener process use $\sqrt{dt}$ instead of $dt$? Why does simulation of random walk in continuous-time not occur as expected?

Essentially because the variance scales quadratically, e.g. $\operatorname{Var}(tX) = t^2\operatorname{Var}(X)$. I'm only going to talk about discrete time processes, rather than something like ...
• 1,578

### Markov Property of a Ito Process

Rather than bringing in $B_{t+h}-B_h$, consider using $B_{t+h}-B_t$: Because $X^x_{t+h} = X^x_t\cdot\exp(ch+\alpha(B_{t+h}-B_t))$, and $B_{t+h}-B_t$ is independent of $\mathcal F_t$ with the same ...
• 26.1k
Accepted

### In a discrete-time Markov chain, is the probability of not being able to get from a state $u$ to $v$ a rational number?

If the transition probabilities are all rational, and the Markov chain is finite, then essentially all probabilities describing the Markov chain's behavior will also be rational, and that extends to ...
• 143k
The converse cannot hold, take a process with trajectories that are equal to a random constant $Z$, so $X_s \equiv Z, s \in T$. Then knowing $X_t$ for some $t \in T$, you know $X_s$ for all times $s \... 1 vote ### Requirements for a Markov chain to converge to its stationary distribution. For an example, see my comment on the discrete time Markov chain (DTMC) with$S=\{1,2\}$and$P_{12}=P_{21}=1$, which is irreducible, recurrent, and periodic and has a solution$\pi = (1/2,1/2)$to$\...
We can show this by induction: for $i=1$ this result is obvious. So suppose $p_i = p_1^i$ for some $i\ge 1$. We want to show $p_{i+1} = p_1p_i$. Define a stopping time $\tau$ to be the first time our ...