I'm self-studying probability theory and struggling with understanding Markov chains on uncountable state spaces, notably I would like to solve the following exercise from this book.

$\textbf{15.2.3.}$ Consider the Markov chain on the real line $\mathbb{R}$, where $P(x, \cdot) = N(\frac{x}{2}, \frac{3}{4})$ for each $x \in \mathbb{R}$. Show that $\pi(\cdot) = N(0,1)$ is a stationary distribution for this chain. And how does one proceed in showing the chain is irreducible with respect to Lebesgue measure?

$P(x,A)$ is the probability that the chain will jump to a subset $A$ being at a point $x$.

In case of finite state space, I seem to have understood it as it boils down to solving a system of equalities. But I don't really see how it works when integrals are involved.

Could anyone offer some illuminating hints? I'll try to work out the rest myself. Thanks in advance.

  • $\begingroup$ The link for the book is broken. I am curious about which book it is. Could you give me the reference please? $\endgroup$ – Gilles Bonnet Feb 2 '15 at 14:54

Since you are already given an stationary distribution, you should only check it. For the definition of the stationary distribution you should have $$ \pi(A) = \int\limits_{\mathbb R}P(y,A)\pi(dy) $$ for any measurable set $A$. So you know that $$ P(y,A) = \int\limits_{A}\frac1{\sigma\sqrt{2\pi}}\mathrm e^{-\frac{(z-y/2)^2}{2\sigma^2}}\,dz. $$ Where $\sigma^2 = 3/4$.

So you should show that $$ \int\limits_{A}\frac1{\sqrt{2\pi}}\mathrm e^{-\frac{y^2}{2}}\,dy = \int\limits_{\mathbb R}\frac1{\sqrt{2\pi}}\mathrm e^{-\frac{y^2}{2}}\,dy\int\limits_{A}\frac1{\sigma\sqrt{2\pi}}\mathrm e^{-\frac{(z-y/2)^2}{2\sigma^2}}\,dz $$

Let as work a bit with the right-hand side: $$ \frac1{2\sigma\pi}\int\limits_{\mathbb R}\mathrm \,dy\int\limits_{A}\mathrm e^{-\frac{(z-y/2)^2}{2\sigma^2}+y^2/2}\,dz = \frac1{2\sigma\pi}\int\limits_{\mathbb R}\mathrm \,dy\int\limits_{A}\exp\left\{-\frac{(z-y/2)^2+\sigma^2y^2}{2\sigma^2}\right\}\,dz $$ $$ = \frac1{2\sigma\pi}\int\limits_{\mathbb R}\mathrm \,dy\int\limits_{A}\exp\left\{-\frac{z^2-zy+y^2}{2\sigma^2}\right\}\,dz= \frac1{2\sigma\pi}\int\limits_{\mathbb R}\mathrm \,dy\int\limits_{A}\exp\left\{-\frac{(y-z/2)^2+\sigma^2z^2}{2\sigma^2}\right\}\,dz $$ and here we change the integrals (do you know why is it allowed?) $$ = \frac{1}{\sqrt{2\pi}} \int\limits_{A}\mathrm e^{-z^2/2}\,dz\int\limits_{\mathbb R}\frac{1}{\sigma\sqrt{2\pi}}\mathrm e^{-\frac{(y-z/2)^2}{2\sigma^2}}\,dy = \frac{1}{\sqrt{2\pi}} \int\limits_{A}\mathrm e^{-z^2/2}\,dz = \pi(A) $$ since the internal integral is always $1$ due to the fact it's an integration ofPDF on the whole state space.

About the irreducibility: we should show that if $\lambda(A)>0$ then the return probability $L(x,A)>0$ for all $x\in \mathbb R$. The simple estimation $L(x,A)\geq P(x,A)>0$ for all $A:\lambda(A)>0$ will lead you to the desired answer.

  • $\begingroup$ Thank you! This is a great answer. I was thinking of something quite similar but having it written fully in front of me (and with square completion!) is very reassuring. $\endgroup$ – johnny Sep 29 '11 at 14:05
  • $\begingroup$ @johnny: you're welcome. $\endgroup$ – Ilya Sep 29 '11 at 14:10
  • $\begingroup$ @johnny: did you like the book you cited? $\endgroup$ – Ilya Sep 29 '11 at 14:13
  • 1
    $\begingroup$ I actually loved it. The book is really well written in terms of rigor (as its name suggest) but also in its style. I believe it is a compilation of lecture notes and you can really feel it, there is always intuition and interesting examples that come along. $\endgroup$ – johnny Sep 29 '11 at 14:25
  • $\begingroup$ @johnny: thank you very much. $\endgroup$ – Ilya Sep 29 '11 at 14:27

Just for fun, let's prove the uniqueness of $\pi$ using characteristic functions. For the given transition kernel, the characteristic function $\hat\pi$ of any invariant probability measure $\pi$ satisfies $$\hat\pi(t)=\hat\pi(t/2) \exp\left(-{t^2\over 2} [3/4]\right).$$ Substituting $t/2$ for $t$ in this equation, and repeating we discover that, for every $n\geq 0$, $$\hat\pi(t)=\hat\pi(t/2^n) \exp\left(-{t^2\over2} [1-1/4^n]\right).$$ Now let $n\to\infty$ and conclude that $\hat\pi(t)=\exp(-t^2/2)$, that is, $\pi$ must be a standard normal distribution.

  • 1
    $\begingroup$ Awesome, Byron! $\endgroup$ – Ilya Sep 29 '11 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.