How is the Kullback-Leibler distance between probability measures well defined Let $\Delta$ be the set of all probability measures on the measurable space $(S,\mathcal{B})$. Assume that all the measures in $\Delta$ are dominated by a measure $\mu$ on $(S,\mathcal{B})$ and let $\psi_Q = \dfrac{dQ}{d\mu}$ denote the Radon-Nikodym derivative of $Q$ w.r.t $\mu$.
Given $\mu$, we can define $$K_{Q:P} = \int \psi_P \log \left (\dfrac{\psi_P}{\psi_Q} \right) d\mu.$$
It is already known that $K_{Q:P} \geq 0$. We can use either Jensen's inequality or Gibbson inequality to prove this.
In one of the papers that I was going through (this paper), the author says that 
"the quanity $K_{Q:P}$ itself is well defined and finite if and only if the function $\psi_P \log \left (\dfrac{\psi_P}{\psi_Q} \right)$ is integrable w.r.t $\mu$. Integrability yields in turn the relation $\psi_Q(x) > 0$ for $P$-almost all $x \in S$, which means $P$ is absolutely continuous w.r.t $Q$."
I am not able to see how the integral being finite will imply and will be implied by the absolute continuity of the measures. Because it is possible that the integral is $\infty$.
 A: You're right, absolute-continuity of the measures alone doesn't imply that the KL divergence exists. Here is my favorite counterexample:

Consider the two probability measures $P,Q$ on $[0,1]$ with pdfs: $f_P=\mathbf{1}_{[0,1]}$ and $f_Q(x)=c\cdot e^{-1/x}\cdot \mathbf{1}_{(0,1]}(x)$ where $c$ is whatever constant will make $Q([0,1])=1.$ Now both $P\ll Q$ and $Q \ll P$ and $P,Q\ll$ (Lebesgue) but $D(Q \| P)$ is infinite, therefore not integrable.

But that's not what they're saying. They are saying that if the absolute-continuity condition holds and the integral is finite, then the KL divergence exists.
In summary all they mean by that is that "The KL divergence is well defined and finite iff all the components of its definition well defined and finite"
A: Note that the function $x\log(x)$ is well defined for all $x\in \mathbb{R}_{+}\setminus \{0\}$. However, in order to maintain right-continuity at $x=0$, it is conventional to define $0\log(0)=\lim\limits_{x\downarrow 0}x\log(x)$.
Now, $\psi_{P}\log\left(\frac{\psi_{P}}{\psi_{Q}}\right)$ is finite if and only if either of the following two conditions is met:
(i) $\psi_{P}\geq0$ and $\psi_{Q}>0$, or
(ii) $\psi_{P}=0=\psi_{Q}$ (where the convention is that $0\log\left(\frac{0}{0}\right)=0$).
In other words, what this suggests is that the support of the measure ${P}$ should be contained in the support of the measure ${Q}$. Put in another way, there cannot be an $s\in S$ for which $\psi_{P}(s)>0$ but $\psi_{Q}(s)=0$. This condition is equivalent to saying "if $\psi_{Q}(s)=0$ for some $s\in S$, then $\psi_{P}(s)=0$", which is equivalently saying that $P<<Q$.
