Correspondence between AB-divergence and Kullback-Leibler divergence

Question

I'm reading up on AB-divergence (alpha-beta-divergence) based mainly on the exposition given in Chichoki et al. (2011), "Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization". I'm particularly puzzled by its correspondence to the standard Kullback-Leibler divergence for parameter values $\alpha = 1$ and $\beta = 0$. Here are the relevant definitions from the paper, my questions follow below

Def. (AB-Divergence) For positive measures $P$ and $Q$:

$$D^{(\alpha,\beta)}_{AB}(P||Q) = -\frac{1}{\alpha\beta} \sum_i (p_{i}^{\alpha} q_i^\beta - \frac{\alpha}{\alpha+\beta} p_i^{\alpha+\beta} - \frac{\beta}{\alpha + \beta} q_i^{\alpha+\beta}), \text{ for } \alpha+\beta\neq 0$$

Reduction to Kullback-Leibler divergence given by $\alpha = 1$ and $\beta = 0$

$$D_{AB}^{(1,0)} (P||Q) = D_{KL}(P||Q) = \sum_i (p_i ln \frac{p_i}{q_i} - p_i + q_i)$$

(1) Can someone walk me through the main steps to get from AB-divergence to KL for these parameter values? I can't wrap my head around it.

(2) Assuming the reduction to be correct (the above is the only paper that makes this correspondence explicit), why does this correspond to KL? As far as I'm aware, the last bit, $- p_i + q_i$, is not part of the standard definition of KL. In fact, assuming $P = Q$ would yield, for all $i$

$$p_i ln \frac{p_i}{q_i} - p_i + q_i = p_i 0 - n = 0 -n, n\geq 0$$

which is (a) negative and (b) not $0$ as one would like to have it with identical distributions.

For clarity, I'm only familiar with KL being defined as

Def. (KL-Divergence) $$D_{KL}(P||Q) = \sum_i p_i ln \frac{p_i}{q_i}$$

Thank you very much!

Answer 1

AB-Divergence in the original form is not defined for $\beta=0$ because $\beta$ appears in the denominator. Instead, we can define $$D_{AB}^{1,0}(P||Q)=\lim_{\beta\to 0}D_{AB}^{1,\beta}(P||Q).$$

Let's calculate the RHS. \begin{align*} D_{AB}^{1,\beta}(P||Q) &=-\frac1{\beta}\sum_i(p_iq_i^{\beta}-\frac1{1+\beta}p_i^{1+\beta}-\frac{\beta}{1+\beta}q_i^{1+\beta})\\ &=-\frac1{\beta}\sum_i\left(p_i(q_i^{\beta}-p_i^{\beta})+\frac{\beta}{1+\beta}(p_i^{1+\beta}-q_i^{1+\beta})\right)\\ &=-\sum_ip_i\frac{q_i^{\beta}-p_i^{\beta}}{\beta}-\frac1{1+\beta}\sum_i (p_i^{1+\beta}-q_i^{1+\beta}). \end{align*}

Using L'Hôpital's rule, $$\lim_{\beta\to 0}\frac{q_i^{\beta}-p_i^{\beta}}{\beta}= \lim_{\beta\to 0}\frac{q_i^{\beta}\ln q_i-p_i^{\beta}\ln p_i}{1} =\ln q_i-\ln p_i=\ln\frac{q_i}{p_i}.$$

Therefore, \begin{align*} D_{AB}^{1,0}(P||Q)&=\lim_{\beta\to 0}D_{AB}^{1,\beta}(P||Q)\\ &=-\sum_i p_i\ln\frac{q_i}{p_i}-\sum_i(p_i-q_i)\\ &=\sum_i \left(p_i\ln\frac{p_i}{q_i}-p_i+q_i\right). \end{align*}

This is equivalent to the above-mentioned definition of KL-divergence when $P$ and $Q$ are probability measures. In that case, $\sum_i p_i=\sum_i q_i=1$ and $\sum_i (-p_i+q_i)=0$ hold.