# Markov chain with uncountable state space

I'm self-studying probability theory and struggling with understanding Markov chains on uncountable state spaces, notably I would like to solve the following exercise from this book.

$\textbf{15.2.3.}$ Consider the Markov chain on the real line $\mathbb{R}$, where $P(x, \cdot) = N(\frac{x}{2}, \frac{3}{4})$ for each $x \in \mathbb{R}$. Show that $\pi(\cdot) = N(0,1)$ is a stationary distribution for this chain. And how does one proceed in showing the chain is irreducible with respect to Lebesgue measure?

$P(x,A)$ is the probability that the chain will jump to a subset $A$ being at a point $x$.

In case of finite state space, I seem to have understood it as it boils down to solving a system of equalities. But I don't really see how it works when integrals are involved.

Could anyone offer some illuminating hints? I'll try to work out the rest myself. Thanks in advance.

• The link for the book is broken. I am curious about which book it is. Could you give me the reference please? – Gilles Bonnet Feb 2 '15 at 14:54

Since you are already given an stationary distribution, you should only check it. For the definition of the stationary distribution you should have $$\pi(A) = \int\limits_{\mathbb R}P(y,A)\pi(dy)$$ for any measurable set $A$. So you know that $$P(y,A) = \int\limits_{A}\frac1{\sigma\sqrt{2\pi}}\mathrm e^{-\frac{(z-y/2)^2}{2\sigma^2}}\,dz.$$ Where $\sigma^2 = 3/4$.

So you should show that $$\int\limits_{A}\frac1{\sqrt{2\pi}}\mathrm e^{-\frac{y^2}{2}}\,dy = \int\limits_{\mathbb R}\frac1{\sqrt{2\pi}}\mathrm e^{-\frac{y^2}{2}}\,dy\int\limits_{A}\frac1{\sigma\sqrt{2\pi}}\mathrm e^{-\frac{(z-y/2)^2}{2\sigma^2}}\,dz$$

Let as work a bit with the right-hand side: $$\frac1{2\sigma\pi}\int\limits_{\mathbb R}\mathrm \,dy\int\limits_{A}\mathrm e^{-\frac{(z-y/2)^2}{2\sigma^2}+y^2/2}\,dz = \frac1{2\sigma\pi}\int\limits_{\mathbb R}\mathrm \,dy\int\limits_{A}\exp\left\{-\frac{(z-y/2)^2+\sigma^2y^2}{2\sigma^2}\right\}\,dz$$ $$= \frac1{2\sigma\pi}\int\limits_{\mathbb R}\mathrm \,dy\int\limits_{A}\exp\left\{-\frac{z^2-zy+y^2}{2\sigma^2}\right\}\,dz= \frac1{2\sigma\pi}\int\limits_{\mathbb R}\mathrm \,dy\int\limits_{A}\exp\left\{-\frac{(y-z/2)^2+\sigma^2z^2}{2\sigma^2}\right\}\,dz$$ and here we change the integrals (do you know why is it allowed?) $$= \frac{1}{\sqrt{2\pi}} \int\limits_{A}\mathrm e^{-z^2/2}\,dz\int\limits_{\mathbb R}\frac{1}{\sigma\sqrt{2\pi}}\mathrm e^{-\frac{(y-z/2)^2}{2\sigma^2}}\,dy = \frac{1}{\sqrt{2\pi}} \int\limits_{A}\mathrm e^{-z^2/2}\,dz = \pi(A)$$ since the internal integral is always $1$ due to the fact it's an integration ofPDF on the whole state space.

About the irreducibility: we should show that if $\lambda(A)>0$ then the return probability $L(x,A)>0$ for all $x\in \mathbb R$. The simple estimation $L(x,A)\geq P(x,A)>0$ for all $A:\lambda(A)>0$ will lead you to the desired answer.

• Thank you! This is a great answer. I was thinking of something quite similar but having it written fully in front of me (and with square completion!) is very reassuring. – johnny Sep 29 '11 at 14:05
• @johnny: you're welcome. – Ilya Sep 29 '11 at 14:10
• @johnny: did you like the book you cited? – Ilya Sep 29 '11 at 14:13
• I actually loved it. The book is really well written in terms of rigor (as its name suggest) but also in its style. I believe it is a compilation of lecture notes and you can really feel it, there is always intuition and interesting examples that come along. – johnny Sep 29 '11 at 14:25
• @johnny: thank you very much. – Ilya Sep 29 '11 at 14:27

Just for fun, let's prove the uniqueness of $\pi$ using characteristic functions. For the given transition kernel, the characteristic function $\hat\pi$ of any invariant probability measure $\pi$ satisfies $$\hat\pi(t)=\hat\pi(t/2) \exp\left(-{t^2\over 2} [3/4]\right).$$ Substituting $t/2$ for $t$ in this equation, and repeating we discover that, for every $n\geq 0$, $$\hat\pi(t)=\hat\pi(t/2^n) \exp\left(-{t^2\over2} [1-1/4^n]\right).$$ Now let $n\to\infty$ and conclude that $\hat\pi(t)=\exp(-t^2/2)$, that is, $\pi$ must be a standard normal distribution.

• Awesome, Byron! – Ilya Sep 29 '11 at 16:40