KL Divergence between the sums of random variables. The relative entropy or Kullback–Leibler distance between
two probability density functions $g(x)$ and $f(x)$ is defined as
$$D(g\|f) = \int_{x} g(x)\log\frac{g(x)}{f(x)}  dx .$$
We have two random variables $V$ and $W$, 
\begin{equation*}
\begin{split}
&V=X_1+X_2, \text{where}\ X_1\sim g(x), X_2\sim f(x)\ \text{are independent},\\
&W=X_3+X_4, \text{where}\ X_3, X_4\sim f(x)\ \text{are independent}.
\end{split}
\end{equation*}
It is easy to show that 
\begin{equation*}
\begin{split}
&V\sim G(x)=(g\ast f)(x),\\
&W\sim F(x)=(f\ast f)(x),
\end{split}
\end{equation*}
where $(g\ast f)(x) = \int g(\tau)f(x-\tau)d\tau$ is the convolution of $g$ and $f$.
The questions are:


*

*Is it true that $D(g\|f)> D(G\|F)?$

*Is it true that $\frac{1}{2}D(g\|f)> D(G\|F)?$
If we can prove 2, 1 is obviously true. 
They are true for Poisson and Gaussian distributions, however, I can't prove for the general cases.  
 A: I will reason in the domain of discret probabilities. As discret distributions are limits of continuous ones, I think we can prove that the result are valid in the domain of continuous densities.
Proof of 1
N.B strict inequality is false. Equality is evident when f=g
We note $g_i$ and $f_i$ the probabilities of g and f and $h_{ij}$ the conditional probabilities of $P(V / X_1)$
We just need the fact that V and W have the same conditional probabilities :
$P(V / X_1)=P(W / X_3)$
$$D(G \parallel F)=\sum_i{(\sum_j{g_jh_{ij}})}\log(\frac{\sum_k{}g_kh_{ik}}{\sum_k{}f_kh_{ik}})$$
We then use the Log-sum Inequality (cf https://en.wikipedia.org/wiki/Log_sum_inequality)
$$\log(\frac{\sum_k{}g_kh_{ik}}{\sum_k{}f_kh_{ik}}) \le \frac{\sum_k{g_kh_{ik}\log(\frac{g_k}{f_k})}}{\sum_k{g_kh_{ik}}}$$
So 
$$D(G \parallel F)\le \sum_{i,k}g_kh_{ik}\log(\frac{g_k}{f_k})=\sum_k[g_k\log(\frac{g_k}{f_k})(\sum_ih_{ik})]=\sum_kg_k\log(\frac{g_k}{f_k})=D(g\parallel f)$$
Counter exemple showing that 2 is not true
Suppose Bernouilli distributions for each $X_i$ :
$P_f(X=0)=1-\epsilon$ 
$P_f(X=1)=\epsilon$ 
and the opposite for g :
$P_g(X=0)=\epsilon$ 
$P_g(X=1)=1-\epsilon$ 
We have :
$P_G(V=0)=P_G(V=2)=\epsilon-\epsilon^2$ 
$P_G(V=1)=1-2\epsilon+2\epsilon^2$
$P_F(W=0)=1-2\epsilon+\epsilon^2$
$P_F(W=1)=2\epsilon-2\epsilon^2$
$P_F(W=2)=\epsilon^2$
It is easy to show that for small $\epsilon$ :
$D(g \parallel f) \sim -\log(\epsilon)$ and $D(G \parallel F) \sim -\log(\epsilon)$
So $D(g \parallel f) \sim D(G \parallel F)$
Therefore, one can find an epsilon so that $\frac 1 2 D(g \parallel f) < D(G \parallel F)$
A: The first inequality is a simple consequence of the chain rule for KL divergences with an additive noise "channel" where $X_2$ (or $X_4$) acts as noise.
