Gibbs' inequality; continuous case I'm reading alternative proof of Gibbs' inequality written in wikipedia, which states that

Suppose that $P=\{p_1,...,p_n\}$ be probability distribution.
Then for any other probability distribution $Q=\{q_1,...,q_n\}$, $-\sum p_i\log{p_i} \leq -\sum p_i\log{q_i}$

I have two question.
First, the proof uses Jensen's inequality to claim that $\sum p_i \log \frac{q_i}{p_i} \leq \log \sum p_i \frac{q_i}{p_i}$.
But why does it hold? I think Jensen's inequality just says that $\sum \log{(p_i \frac{q_i}{p_i})} \leq \log \sum p_i \frac{q_i}{p_i}$
Second, can we apply Gibbs' inequality in the continuous case?
i.e., does $-\int f(x)\log{f(x)} dx \leq -\int f(x)\log{g(x)} dx$ still holds for probability density function $f, g$?
I think since we can apply Jensen's inequality in the continuous case, we can still argue that the continuous case holds. Nevertheless, I cannot find any mention of Gibbs' inequality in the continuous case; they only deal with the discrete case. Is there any problem to deal with the continuous case? Or can I use Gibbs' inequality in the continuous case as I write?
Thanks.
 A: Partial answer for the first part using "elementary stuff" that OP has asked for in comments:




symbol
explantion




$p_i$
weights


$\log$
concave function


$q_i/p_i$
points in the domain where the concave function is defined




In the quoted inequality $$\sum p_i \log \frac{q_i}{p_i} \leq \log \sum p_i \frac{q_i}{p_i},$$

*

*LHS: weighted average of $\log(q_i/p_i)$'s.

*RHS: $\log$ of weighted average of $q_i/p_i$'s.

Note that $\log$ is concave, so Jensen's inequality would give the desired result.
The second part is addressed in the linked blog article, and I'm not going retype his/her arguments.
A: This is a very long date post, but I would like to answer because I am currently facing the same problem. The continuous version of Gibb's inequality is a consequence of Jensen's inequality.
Let $f(x)$ be the true density of $X$ and $g(x)$ an arbitrary density function. Applying Jensen's inequality to $\dfrac{g(x)}{f(x)}$ and since the function $\log$ is concave, we have $E\left(\log\left(\dfrac{g(x)}{f(x)}\right)\right) \leq \log\left(E\left(\dfrac{g(x)}{f(x)}\right)\right) = \log\left(\int \dfrac{g(x)}{f(x)} f(x) dx \right) = \log\Big(\underbrace{\int g(x) dx}_{=1}\Big) $ = 0. Besides, $E\left(\log\left(\dfrac{g(x)}{f(x)}\right)\right) = E\left(\int\log\left(\dfrac{g(x)}{f(x)}\right) f(x) dx\right) = \int f(x)\log\left(g(x)\right)  dx - \int f(x)\log\left(f(x)\right)  dx$. As a result,
$$\int f(x)\log\left(f(x)\right) \geq \int g(x)\log\left(f(x)\right),$$
which is the continuous version of the Gibbs' Inequality
