2
$\begingroup$

I have the following continuous time Markov chain $X = (X_t : t \geq 0)$ with generator matrix given by $g_{i,i+1} = \lambda_i$ for $i \geq 0$, $g_{i, 0} = \lambda_i \rho_i$ for $i > 0$, and $g_{ij} = 0$ for $j \neq 0, i, i+1$. Intuitively this chain increases by one and then at some point gets sent back to the origin. I want to show that this chain is transient if and only if $\prod_i (1 + \rho_i) < \infty$.

A first thought is to write down the jump matrix and try proving that the jump chain is transient (as that would then prove that the continuous chain is transient, giving us the $\Leftarrow$ direction). I have calculated the jump matrix as $p_{0, 1} = 1$, $p_{i, 0} =\frac{\rho_i}{\rho_i + 1}$, $p_{i,i+1} = \frac{1}{\rho_i + 1}$ and $p_{i,j} = 0$.

I do not know how to show that the jump chain is transient. This is because there seems to be many ways that a chain could return to zero so the probability that the chain eventually returns to zero given it starts at zero is not easy to calculate.

I also do not know how to show the $\Rightarrow$ direction (which I assume requires some kind of a direct proof).

How can this be done?

$\endgroup$

1 Answer 1

2
$\begingroup$

Say we start at state $0$. What events need to happen for us to never return to $0$? For this particular chain, there's only one path that doesn't return.

  • From $0$, we need to step to $1$ (prob. $1$).
  • From $1$, we need to step to $2$ (prob. $\frac{1}{1 + \rho_1}$).
  • ...
  • From $i$, we need to step to $i+1$ (prob. $\frac{1}{1 + \rho_i}$).
  • ...

So never returning to $0$ has the same probability as a sequence of infinitely many independent weighted coin flips all coming up heads, where the heads probability of coin $i$ is $\frac{1}{1 + \rho_i}$.

I'll let you think about how to formalize the statement above, and how to prove both directions once you have it. (You will need continuity of probability to reason about the infinite intersection of coin-is-heads events.)

$\endgroup$
1
  • $\begingroup$ Thank you, I believe I have solved it with your hint. As $X$ is transient the jump chain is transient $\iff \prod_{i} (1+\rho_i) < \infty$ which can be shown using your hint, is that right? Thanks! And for my $\iff$ statement there, its proven by calculating $\mathbb{P}(\bigcap \{\text{toss i is heads}\}) \rightarrow p$ where $0 < p < 1$ which precisely means the jump chain has to be transient. $\endgroup$ Commented Apr 20 at 11:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .