New answers tagged

2 votes
Accepted

Prove that $P(X_{n+m}=j|X_0=i_0,\dots, X_n=i)=p_{ij}^{(m)}$

I will sketch the proof of the case $m=2$; the general case is analogous. Let $E_{n,i} = \{X_0=i_0, \ldots, X_n=i\}$. \begin{align} P(X_{n+2} = j \mid E_n) &= \sum_{k} P(X_{n+2} = j, X_{n+1} = k \...
angryavian's user avatar
1 vote

A question about Markov chain's definition

It is not necessary: if you know that $$P(Z_{n+1}=y \mid Z_0=x_0,\dots,Z_{n-1}=x_{n-1},Z_n=x)=P(Z_1=y \mid Z_0=x)$$ then you can show that both of those are equal to $P(Z_{n+1}=y\mid Z_n = x)$. To ...
Misha Lavrov's user avatar
0 votes

Exercise on irreducible positive recurrent markov chain

1. Let $N_n(A)$ be the number of visits to $A$ by time $n$; that is, $N_n=\sum_{j=1}^n 1_A(X_j)$. What can you say about $N_n(A)/n$ as $n\to\infty$? 2. Observe that, for each $n$ and $k$, $T_k^A\le n$...
John Dawkins's user avatar
  • 26.1k
0 votes

Markov process characterization and application of monotone class theorem

It's my first time answering questions in stochastic processes, so please correct me if I'm wrong. Following @TheBridge's comment, we can use the Functional Monotone Class Lemma (Theorem 2 in ...
isomorphicdude's user avatar
1 vote

Properties of a transient state in a Markov Chain

As you have pointed out, $\sum\limits_{n=1}^{\infty}P_{jj}^{n}<\infty$, and that $\mathbb{P}_{ij}(s)=\mathbb{F}_{ij}(s)\mathbb{P}_{jj}(s)$, what you need follows directly from Abel's theorem, since ...
Sparkle-Lin's user avatar
3 votes
Accepted

Why does the Wiener process use $\sqrt{dt}$ instead of $dt$? Why does simulation of random walk in continuous-time not occur as expected?

Essentially because the variance scales quadratically, e.g. $\operatorname{Var}(tX) = t^2\operatorname{Var}(X)$. I'm only going to talk about discrete time processes, rather than something like ...
daisies's user avatar
  • 1,578
0 votes

Markov Property of a Ito Process

Rather than bringing in $B_{t+h}-B_h$, consider using $B_{t+h}-B_t$: Because $X^x_{t+h} = X^x_t\cdot\exp(ch+\alpha(B_{t+h}-B_t))$, and $B_{t+h}-B_t$ is independent of $\mathcal F_t$ with the same ...
John Dawkins's user avatar
  • 26.1k
2 votes
Accepted

In a discrete-time Markov chain, is the probability of not being able to get from a state $u$ to $v$ a rational number?

If the transition probabilities are all rational, and the Markov chain is finite, then essentially all probabilities describing the Markov chain's behavior will also be rational, and that extends to ...
Misha Lavrov's user avatar
0 votes

Equivalent definition of Markov property

The converse cannot hold, take a process with trajectories that are equal to a random constant $Z$, so $X_s \equiv Z, s \in T$. Then knowing $X_t$ for some $t \in T$, you know $X_s$ for all times $s \...
Moritz Schauer's user avatar
1 vote

Requirements for a Markov chain to converge to its stationary distribution.

For an example, see my comment on the discrete time Markov chain (DTMC) with $S=\{1,2\}$ and $P_{12}=P_{21}=1$, which is irreducible, recurrent, and periodic and has a solution $\pi = (1/2,1/2)$ to $\...
Michael's user avatar
  • 24k
0 votes

Random walk hitting probability

We can show this by induction: for $i=1$ this result is obvious. So suppose $p_i = p_1^i$ for some $i\ge 1$. We want to show $p_{i+1} = p_1p_i$. Define a stopping time $\tau$ to be the first time our ...
raj's user avatar
  • 251

Top 50 recent answers are included