# Tag Info

3

Hint: Define the random variable with $Y_n \in \{0,1\}$ (indicator variable with $2$ states) as follows $$Y_n=\begin{cases}1,& X_n=6\\0,&X_n\neq6\end{cases}$$ with transition matrix $$\mathbf P_{(1)}=\begin{array}{r|cc|r}&0&1&\\\hline0&\frac{4}{5}&\frac{1}{5}\\1&1&0 \end{array}$$ Initially $Y_0=1$. You want to find the ...

2

A finite Markov chain always has at least one steady-state distribution. If the transition matrix is $A$, each column of $A-I$ sums to $0$, so $A-I$ doesn't have full rank, and there is at least one nontrivial solution to $Ax=x$. On the other hand a Markov chain with an infinite state space doesn't have to have a steady-state distribution. For example, ...

1

An elementary argument we can make is as follows: Define by $d_n$ the expected number of deaths occurring before we reach state $n+1$ given that we start in state $n$. We clearly have $d_0=0$. Further we can write an expression for $d_n$ noting that, with, probability $\lambda_n$, we will have no deaths before reaching state $n+1$, and with probability ...

1

I was able to figure it out eventually... 1. \begin{align*} &\sum_{i \in J} \left|\frac{V_i(n)}{n}-\pi_i\right| + \sum_{i \notin J} \left(\frac{V_i(n)}{n}+\pi_i\right)\\ &= \sum_{i \in J} \left|\frac{V_i(n)}{n}-\pi_i\right| + \left(1-\sum_{i \in J} \frac{V_i(n)}{n}\right)+\sum_{i \notin J}\pi_i & \sum_{i \in I} V_i(n)=n\\ &= \sum_{i \in J} ...

1

For future reference, I found a nice source myself: http://www3.ntu.edu.sg/home/nprivault/papers/greifswald_potential.pdf Sections 1 and 2 are on potential theory. Sections 3 and 4 are on probability. Section 5 presents a number of the sorts of connections I was looking for here. A bit of overlap is splashed between the first four sections. It is a very ...

1

Let $f: \mathbb{R}^2 \to \mathbb{R}$ be a bounded continuous function. In order to show that $$M_t := (I_t,W_t) := \left( \int_0^t W_s \, ds, W_t \right)$$ is Markov, we have to show that there exists $g: \mathbb{R}^2 \to \mathbb{R}$ such that $$\mathbb{E}^x(f(M_t) \mid \mathcal{F}_s) = g(M_s)$$ for any $s \leq t$. To this end, write \begin{align*} ...

Only top voted, non community-wiki answers of a minimum length are eligible