Let $\mu$ and $\lambda$ be probability measures on a measurable space $(X, \Sigma)$. In my experience, the usual definition of the Kullback-Liebler divergence of $\mu$ with respect to $\lambda$ is $$ \tag{1} \label{kl def} \operatorname{KL}(\mu \| \lambda) = \begin{cases} \int_X \log\left(\frac{d\mu}{d\lambda}\right) \, d\mu, & \text{if $\mu \ll \lambda$ and $\log\left(\frac{d\mu}{d\lambda}\right) \in L^1(\mu)$,} \\ \infty, & \text{otherwise.} \end{cases} $$ While reading some machine learning theory literature, I encountered the following inequality, attributed to Donsker and Varadhan, which is valid at least for bounded, $\Sigma$-measurable functions $\Phi : X \to \mathbb{R}$: $$ \tag{2} \label{kl ineq} \int_X \Phi \, d\mu \leq \operatorname{KL}(\mu \| \lambda) + \log\int_X \exp(\Phi) \, d\lambda. $$ This led me to a 1983 paper by Donsker and Varadhan (see References below), in which they define the entropy of $\mu$ with respect to $\lambda$ by $$ \tag{3} \label{dv def} h(\lambda : \mu) = \inf\left\{c \in \mathbb{R} : \int_X \Phi \, d\mu \leq c + \log\int_X \exp(\Phi) \, d\lambda \quad\text{for all $\Phi \in \mathscr{B}(\Sigma)$} \right\}, $$ where $\mathscr{B}(\Sigma)$ is the space of all bounded, $\Sigma$-measurable functions from $X$ to $\mathbb{R}$.
The paper makes several assertions about this definition. For instance,
- If $X$ is a separable, completely metrizable space and $\Sigma$ is its Borel $\sigma$-algebra, then $\mathscr{B}(\Sigma)$ can be replaced by $C(X)$ in \eqref{dv def}, yielding the same infimum. (Presumably $C(X)$ here is the space of continuous functions on $X$, but not all such functions are necessarily $\mu$-integrable, so maybe the space of compactly supported continuous functions is intended?)
- If $X$ is a separable, completely metrizable space and $\Sigma$ is its Borel $\sigma$-algebra, then $h(\lambda : \mu)$ is lower semicontinuous in $\mu$ in the weak topology.
- (Theorem 2.1) $h(\lambda : \mu) = \operatorname{KL}(\mu \| \lambda)$ (i.e., \eqref{kl def} and \eqref{dv def} define the same quantity).
I'm most interested in the first and last items above, whose proofs can be apparently found in an earlier 1976 paper by Donsker and Varadhan (see References below). However, I was unable to find anything resembling these results in that paper.
Questions
How can I prove the assertions about $h(\lambda : \mu)$ made in the 1983 Donsker-Varadhan paper? In particular, why is $h(\lambda : \mu) = \operatorname{KL}(\mu \| \lambda)$?
For which functions $\Phi$ does \eqref{kl ineq} hold? It certainly holds for all bounded, $\Sigma$-measurable functions by the definition of $h(\lambda:\mu)$, and it holds for non-negative, $\Sigma$-measurable functions by the monotone convergence theorem. Does it hold for all $\mu$-integrable functions?
The machine learning literature also uses the following representation of Kullback-Liebler divergence, which is also attributed to Donsker and Varadhan: $$ \operatorname{KL}(\mu \| \lambda) = \sup_{\Phi \in \mathcal{C}} \left(\int_X \Phi \, d\mu - \log\int_X \exp(\Phi) \, d\lambda\right), $$ where $\mathcal{C}$ is a usually unspecified class of functions (presumably $\mathcal{C} = \mathscr{B}(\Sigma)$ works). This looks like a dual formulation of \eqref{dv def}, but I would appreciate a proof of this as well (in particular, the $\infty - \infty$ case may need to be addressed).
References
Donsker, M.D. and Varadhan, S.R.S. (1976), Asymptotic evaluation of certain Markov process expectations for large time—III. Comm. Pure Appl. Math., 29: 389-461. DOI
Donsker, M.D. and Varadhan, S.R.S. (1983), Asymptotic evaluation of certain markov process expectations for large time. IV. Comm. Pure Appl. Math., 36: 183-212. DOI