# Can we solve KL divergence between Gaussian Mixtures by thinking conditional cases?

I learned that KL divergence between two Gaussian Mixtures is intractable, not easy to solve. But I am wondering if we can solve it by thinking conditional cases?

Let's say, a single multivariate Gaussian and a 2-mixture multivariate Gaussian as shown below. $$g_{A}=Gauss(\mu_A,\sigma_A )$$ $$g_{B+C}=w_B\cdot Gauss(\mu_B,\sigma_B )+w_C\cdot Gauss(\mu_C,\sigma_C )$$ Is KL divergence below correct?

$$D_{KL} \left( g_{A}\right| \left| g_{B+C} \right)=w_B\cdot D_{KL} \left( g_{A}\right| \left| g_{B} \right)+w_C\cdot D_{KL} \left( g_{A}\right| \left| g_{C} \right)$$

seeing weights $w_B$ ,$w_C$ as conditional probabilities, $P(D_{KL}|g_B)$,$P(D_{KL}|g_C)$.

No, the KL divergence $D(p||q)$ is convex in the pair $(p,q)$ so the equation you wrote holds as an inequality: $$D_{KL} \left( g_{A}\right| \left| g_{B+C} \right)\leq w_B\cdot D_{KL} \left( g_{A}\right| \left| g_{B} \right)+w_C\cdot D_{KL} \left( g_{A}\right| \left| g_{C} \right)$$ See this paper by Hershey and Olsen and also this post on related material for approximating the KL between mixtures of distributions.