# measure theory question about densities

Here we have that $\pi$ is the invariant distribution of a Markov chain $K$. Let's say that $\mu=f\pi$, that is, $f$ is the density of $\mu$ and $\mu$ is a probability measure.

Why does $\mu K$ have density $K^* f$? I don't understand where the adjoint comes from because we're not dealing with any inner products here.

-

Three kinds of objects are concerned: functions, measures and transition kernels. One can think about all of them as matrices of different sizes: for Markov chains defined on a finite state space of size $n$, functions have size $1\times n$, measures have size $n\times 1$ and transition kernels have size $n\times n$. Of course, for infinite state spaces, the dimension $n$ is infinite, nevertheless the interpretation as extended matrices is still useful.
For example, if $K$ is a kernel and $f$ is a function, $Kf$ has size $n\times n$ composed with $n\times 1$, this is size $n\times 1$ hence $Kf$ is a function, as it should be. Likewise, if $\mu$ is a measure, $\mu K$ has size $1\times n$ composed with $n\times n$, this is size $1\times n$ hence $\mu K$ is a measure, as it should be. And so on. Relevant formulas are $$(Kf)(x)=\int K(x,\mathrm{d}y)f(y),\quad (\mu K)(\mathrm{d}x)=\int\mu(\mathrm{d}y)K(y,\mathrm{d}x).$$ Let us turn to your question. The setting is that $\pi$ is a stationary measure, that is, $\pi=\pi K$, or $$\pi(\mathrm{d}x)=\int \pi(\mathrm{d}y)K(y,\mathrm{d}x),$$ and that $\mu$ has density $f$ with respect to $\pi$, that is, $$\mu(\mathrm{d}x)=f(x)\pi(\mathrm{d}x).$$ We are concerned with the measure $\nu=\mu K$, that is, $$\nu(\mathrm{d}x)=\int\mu(\mathrm{d}y)K(y,\mathrm{d}x)=\int f(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x),$$ and we want to know the density $g$ of $\nu$ with respect to $\pi$, that is, we want to write $\nu$ as $$\nu(\mathrm{d}x)=g(x)\pi(\mathrm{d}x).$$ The one and only possible function $g$ is $$g(x)=\int f(y)\pi(\mathrm{d}y)\frac{K(y,\mathrm{d}x)}{\pi(\mathrm{d}x)}.$$ Hence $g=K^*f$ for a carefully chosen kernel $K^*$, namely $$K^*(x,\mathrm{d}y)=\frac{K(y,\mathrm{d}x)}{\pi(\mathrm{d}x)}\pi(\mathrm{d}y).$$ One sees that $K^*$ is indeed the adjoint of $K$, as the notation suggests, in the following sense: for every functions $f_1$ and $f_2$ that are square integrable with respect to $\pi$, define their inner product in $L^2(\pi)$ by $$\langle f_1,f_2\rangle_\pi=\int f_1(x)f_2(x)\pi(\mathrm{d}x).$$ Then, Fubini theorem yields $$\langle f_1,K^*f_2\rangle_\pi=\int\int f_1(x)f_2(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x)=\int Kf_1(y)f_2(y)\pi(\mathrm{d}y)=\langle Kf_1,f_2\rangle_\pi,$$ which is the definition of the adjoint of a linear operator.
@Didier: Nice answer! I was referred here by your reply to my stationarity of Markov process question. I wonder about the way you construct the density g of ν with respect to π, by $g(x)=\int f(y)\pi(\mathrm{d}y)K(y,\mathrm{d}x) \frac1{\pi(\mathrm{d}x)}$. Is it part of Radon-Nykodym Theorem? Can I know a little more about this construction? – Tim May 1 '11 at 19:56