$\int f \;d\mu=\int g \;d\mu=1$ and $f \log g \in L^1(\mu)$ implies $\int f \log g \; d\mu\leq \int f \log f \; d\mu$ I'm trying to show the following inequality. Let $f,g:X\to (0,\infty)$ be measurable functions with $\int f\; d\mu=\int g \;d\mu=1$. Show that if $f \log g \in L^1(\mu)$, then
$$\int f \log g \;d\mu\leq \int f \log f \;d\mu$$
This seems like an application of Jensen's inequality but I'm having difficulty applying it correctly. I defined a measure $$d\nu=f\;d\mu$$ and so I need to show that
$$\int  \log g \;d\nu\leq \int  \log f \;d\nu$$
Note that $(X,\nu$) is a probability space. I need help recognizing the convex function $\phi$ and the $\nu-$integrable function $F$ onto which I can apply Jensen's inequality:
$$\phi(\int F\;d\nu)\leq \int \phi(F)\;d\nu$$
Any help with this is appreciated. Thanks for your time.
 A: Special Case. The claim easily follows from Jensen's inequality if we also assume that $f\log f$ is in $L^1(\mu)$:
$$ \int \log\left(\frac{g}{f}\right) \, f \, \mathrm{d}\mu
\overset{\text{(Jensen)}}{\leq} \log \left( \int \frac{g}{f} \, f \, \mathrm{d}\mu \right)
= \log \left( \int g \, \mathrm{d}\mu \right)
= 0. $$

General Case. Now we turn to the general case. If we are only given that $\int f \, \mathrm{d}\mu = 1$, then it is possible to have both $\int \left(f\log f\right)_+ \, \mathrm{d}\mu = \infty$ and $\int \left(f\log f\right)_- \, \mathrm{d}\mu = \infty$:

Example. Let $f \in \mathbb{R} \to (0, \infty)$ be defined by
$$ f(x) = \begin{cases}
\frac{2}{x (\log x)^2}, & \text{if } x \in (0, e^{-4}) \cup (e^4, \infty), \\
0, & \text{elsewhere}.
\end{cases} $$
Then $f$ is a valid PDF such that $f > 1$ on $(0, e^{-4})$ and $0 < f < 1$ on $(e^4, \infty)$. However, it is easy to check that $\int \left(f\log f\right)_+ \, \mathrm{d}\mu = \infty$ and $\int \left(f\log f\right)_- \, \mathrm{d}\mu = \infty$.

So, OP's assumption $f \log g \in L^1(\mu)$ will play a crucial role in ensuring that $\int f \log f \, \mathrm{d}\mu$ is well-defined. More generally, we will prove:

Theorem. Let $f, g : X \to (0, \infty)$ be measurable on the measure space $(X, \mathcal{F}, \mu)$. Suppose

*

*$\int f \, \mathrm{d}\mu = 1$ and $\int g \, \mathrm{d}\mu = 1$, and

*either $\int (f \log g)_- \, \mathrm{d}\mu < \infty$ or $\int (f \log f)_+ \, \mathrm{d}\mu < \infty$.

Then both $\int f \log g \, \mathrm{d}\mu$ and $\int f \log f \, \mathrm{d}\mu$ exist, and we have
$$ \int f \log g \, \mathrm{d}\mu \leq \int f \log f \, \mathrm{d}\mu. $$

Proof. To begin with, we fix a compact subset $K$ of $(0, \infty)$ so that $\mu(\{f, g \in K\}) > 0$. Then, for any compact intervals $I = [a, b]$ and $J = [c, d]$ of $(0, \infty)$ containing $K$, we define the measure $\nu_{I, J}$ on $X$ by
$$ \nu_{I, J}(\mathrm{d}x) = \frac{f(x) \mathbf{1}_{\{ f \in I, g \in J \}}(x) \, \mu(\mathrm{d}x)}{\int f \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu}. $$
Since $ \int f \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu
\geq \int f \mathbf{1}_{\{ f, g \in K \}} \, \mathrm{d}\mu
> 0 $, this is indeed a well-defined probability measure. Moreover, the choice of truncation $\mathbf{1}_{\{ f \in I, g \in J \}}$ guarantees that $\log f$ and $\log g$ are in $L^1(\nu_{I, J})$. So by the Jensen's inequality,
\begin{align*}
\int \log\left(\frac{g}{f}\right) \, \mathrm{d}\nu_{I, J}
\leq \log\left( \int \frac{g}{f} \, \mathrm{d}\nu_{I, J} \right)
= \log\left( \frac{\int g \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu}{\int f \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu} \right). \tag{1}
\end{align*}
Rearranging $\text{(1)}$, we get
\begin{align*}
\int \left( f \log g \right) \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu
&\leq \int \left( f \log f \right) \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu + E_{I, J}, \tag{2}
\end{align*}
where $E_{I,J}$ is defined by
$$ E_{I,J}
= \left( \int f \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu \right) \log\left( \frac{\int g \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu}{\int f \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu} \right). $$
By our choice of $K$, we can check that $E_{I, J}$ is bounded. Let $M$ be a bound of $E_{I, J}$. Now we example two cases:

*

*Case 1. Suppose $\int \left(f \log g\right)_- \, \mathrm{d}\mu < \infty$. Then rearranging $\text{(2)}$, we get
\begin{align*}
\int \left( - f \log f \right) \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu
&\leq \int \left( - f \log g \right) \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu + E_{I, J} \\
&\leq \int (f \log g)_- \, \mathrm{d}\mu + M,
\end{align*}
and so,
\begin{align*}
\int \left(f \log f \right)_- \mathbf{1}_{\{ f \in I, g \in J \}} \, \mathrm{d}\mu
&\leq \int \left(f \log f \right)_+ \mathbf{1}_{\{ f \in I \}} \, \mathrm{d}\mu + \int (f \log g)_- \, \mathrm{d}\mu + M.
\end{align*}
Recall that $I = [a, b]$ and $J = [c, d]$. Assuming WLOG $b > 1$ and then letting $a, c \to 0^+$ and $d \to \infty$, MCT shows that
\begin{align*}
\int \left(f \log f \right)_- \, \mathrm{d}\mu
&\leq \int \left(f \log f \right)_+ \mathbf{1}_{\{ f \leq b \}} \, \mathrm{d}\mu + \int (f \log g)_- \, \mathrm{d}\mu + M
< \infty.
\end{align*}


*Case 2. Suppose $\int \left(f \log f\right)_+ \, \mathrm{d}\mu < \infty$. Then arguing similarly as in Case 1, we can show that $\int \left(f \log g\right)_+ \, \mathrm{d}\mu < \infty$.
Hence, the assumption implies that both $\int \left(f \log f\right) \, \mathrm{d}\mu $ and $\int \left(f \log g\right) \, \mathrm{d}\mu $ exist. Finally, by letting $a, c \to 0^+$ and $b, d \to \infty$ in $\text{(2)}$,
\begin{align*}
\int f \log g \, \mathrm{d}\mu \leq \int f \log f \, \mathrm{d}\mu + \underbrace{ \left( \int f \, \mathrm{d}\mu \right) \log\left( \frac{\int g \, \mathrm{d}\mu}{\int f \, \mathrm{d}\mu} \right) }_{=0}
\end{align*}
and therefore the desired claim follows.
