Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Here we have that $\pi$ is the invariant distribution of a Markov chain $K$. Let's say that $\mu=f\pi$, that is, $f$ is the density of $\mu$ and $\mu$ is a probability measure.

Why does $\mu K$ have density $K^* f$? I don't understand where the adjoint comes from because we're not dealing with any inner products here.

share|cite|improve this question
up vote 9 down vote accepted

It seems part of the trouble here is caused by not defining properly the objects one is talking about, so let us try to be very careful about this.

Three kinds of objects are concerned: functions, measures and transition kernels. One can think about all of them as matrices of different sizes: for Markov chains defined on a finite state space of size $n$, functions have size $1\times n$, measures have size $n\times 1$ and transition kernels have size $n\times n$. Of course, for infinite state spaces, the dimension $n$ is infinite, nevertheless the interpretation as extended matrices is still useful.

For example, if $K$ is a kernel and $f$ is a function, $Kf$ has size $n\times n$ composed with $n\times 1$, this is size $n\times 1$ hence $Kf$ is a function, as it should be. Likewise, if $\mu$ is a measure, $\mu K$ has size $1\times n$ composed with $n\times n$, this is size $1\times n$ hence $\mu K$ is a measure, as it should be. And so on. Relevant formulas are $$ (Kf)(x)=\int K(x,\mathrm{d}y)f(y),\quad (\mu K)(\mathrm{d}x)=\int\mu(\mathrm{d}y)K(y,\mathrm{d}x). $$ Let us turn to your question. The setting is that $\pi$ is a stationary measure, that is, $\pi=\pi K$, or $$ \pi(\mathrm{d}x)=\int \pi(\mathrm{d}y)K(y,\mathrm{d}x), $$ and that $\mu$ has density $f$ with respect to $\pi$, that is, $$ \mu(\mathrm{d}x)=f(x)\pi(\mathrm{d}x). $$ We are concerned with the measure $\nu=\mu K$, that is, $$ \nu(\mathrm{d}x)=\int\mu(\mathrm{d}y)K(y,\mathrm{d}x)=\int f(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x), $$ and we want to know the density $g$ of $\nu$ with respect to $\pi$, that is, we want to write $\nu$ as $$ \nu(\mathrm{d}x)=g(x)\pi(\mathrm{d}x). $$ The one and only possible function $g$ is $$ g(x)=\int f(y)\pi(\mathrm{d}y)\frac{K(y,\mathrm{d}x)}{\pi(\mathrm{d}x)}. $$ Hence $g=K^*f$ for a carefully chosen kernel $K^*$, namely $$ K^*(x,\mathrm{d}y)=\frac{K(y,\mathrm{d}x)}{\pi(\mathrm{d}x)}\pi(\mathrm{d}y). $$ One sees that $K^*$ is indeed the adjoint of $K$, as the notation suggests, in the following sense: for every functions $f_1$ and $f_2$ that are square integrable with respect to $\pi$, define their inner product in $L^2(\pi)$ by $$ \langle f_1,f_2\rangle_\pi=\int f_1(x)f_2(x)\pi(\mathrm{d}x). $$ Then, Fubini theorem yields $$ \langle f_1,K^*f_2\rangle_\pi=\int\int f_1(x)f_2(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x)=\int Kf_1(y)f_2(y)\pi(\mathrm{d}y)=\langle Kf_1,f_2\rangle_\pi, $$ which is the definition of the adjoint of a linear operator.

share|cite|improve this answer
THANK YOU! This helps so much---exactly what I needed. – Clark Kent Mar 20 '11 at 14:36
You are welcome. Tried to help. – Did Mar 20 '11 at 14:48
Really a very good answer – TheBridge Mar 20 '11 at 16:51
@TheBridge Thanks. Funny how everything becomes smooth and natural once one uses proper notations... – Did Mar 20 '11 at 18:09
@Didier: Nice answer! I was referred here by your reply to my stationarity of Markov process question. I wonder about the way you construct the density g of ν with respect to π, by $g(x)=\int f(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x) \frac1{\pi(\mathrm{d}x)}$. Is it part of Radon-Nykodym Theorem? Can I know a little more about this construction? – Tim May 1 '11 at 19:56

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.