3

The continuous case is quite analogous; it is simply $\pi_0 e^{tQ}$ (I think it is customary to put the law on the left and the functions on the right by adjointness). That is because the diffusion has to solve Kolmogorov's forward-backward equations. ($P(t) = e^{tQ}$ is your transition matrix for time $t$, and it satisfies $\frac{\partial P(t)}{\partial t} =...


3

I think the following example can maybe illustrate what is happening. John works at noob corporation. His boss, in typical fashion is annoying. There are three "moods" in which John can be, they are Fired,angry and happy with transitions given by this matrix: $\begin{pmatrix} 1 & 0 & 0 \\ 0.5 & 0.4 & 0.1 \\ 0 & 0.2 & 0.8 \\ \...


2

The $i^\text{th}$ pad can be reached from pads indexed $i-1,i,i+1,...$ with probabilities $\frac1i,\frac1{i+1},\frac1{i+2},...$ respectively. Thus steady state probability of $i^\text{th}$ lily pad,$$p_i=\sum_{k=i-1}^\infty\frac{p_k}{k+1};i\ge1,p_0=0$$ and $\sum_{k=1}^\infty p_k=1$. So $p_i=\frac{p_{i-1}}i+p_{i+1}$, giving us$$p_{n}=p_{n-1}-\frac{p_{n-2}}{n-...


1

Your $N$ matrix should be $$N = (I-Q) = \begin{bmatrix} 1 & -1/3 & -1/3 \\ 0 & 1 & -1/2 \\ 0 & 0 & 1 \end{bmatrix}.$$ Remember, $I$ is the identity matrix, not the all $1$s matrix. The inverse of this is $$(I-Q)^{-1} = \begin{bmatrix} 1 & 1/3 & 1/2 \\ 0 & 1 & 1/2 \\ 0 & 0 & 1 \end{bmatrix},$$ and the sum of ...


1

Hint: $$P(X_5 = 2 \mid X_3 = 0, X_1 = 1) = \sum_{n=0}^2P(X_5 = 2 \mid X_4 = n)P(X_4=n\mid X_3=0)$$


1

You're close. Let $E_j$ be the expected number of steps to get to state $j$ from state $1$. Then $$E_j=E_{j-1}+\frac12(1)+\frac12(1+E_j)\implies E_j=2E_{j-1}+2$$ In order to get to state $j$, we must first get to state $j-1$. There's no reason to multiply by $\frac12$ here. Then half the time, we finish in one more transition, and half the time we begin ...


1

HINTS You can determine $c$ from the fact that $\{X_n\}_n$ is a martingale. Can you write down the condition implied by the definition and imply $c$ from it? $\mathbb{E}[X_3]$ can be computed directly by listing all possible paths in 3 jumps. $\mathbb{E}[X_T]$ can be computed using Doob's Optional Stopping Theorem. That result is one of the major advantages ...


1

States 3 and 5 are the absorbing states of the Markov chain. The question is what is the probability of ending up in each state. There are some simplifications we can make. With probability 1, we will reach state 1. (Equivalently, the probability of never seeing a heads is zero.) If we are in state 2, we will eventually reach state 3 with probability 1. So,...


1

Without loss of generality, let $i=0$. Note that you are dealing with a first passage time (i.e., the first time your random walk hits $0$), and therefore, it is not sufficient to just consider the current state at time $2n$. In the second equation (the one not marked), you are calculating the probability of the following: in the first $2n$ steps, what is ...


Only top voted, non community-wiki answers of a minimum length are eligible