Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Following is a part of an answer which was not resolved when I tried to answer a question in mathoverflow. I thought it would be nice to discuss that here.

Let $P$ and $Q$ be two distinct distributions on a finite set. For $0\le \lambda \le 1$, let $$L(\lambda)=D(P\|R_{\lambda})-D(Q\|R_{\lambda}),$$ where $$R_{\lambda}=\lambda P+(1-\lambda) Q.$$ I want to know the range of $\lambda$ for which $L(\lambda)\ge 0$.

I found that $\frac{d}{d\lambda}L(\lambda)=-\sum \frac{(P(a)-Q(a))^2}{R_{\lambda}(a)}\le 0$. So $L(\lambda)$ is decreasing. Also $L(0)=D(P\|Q)>0$ and $L(1)=-D(Q\|P)<0$. So at some $\lambda$, $L(\lambda)=0$.

The problem would be solved if we can find that $\lambda$. Is there a way to find it?

share|cite|improve this question
You could explain a reason why an expression for the solution $\lambda$ of $L(\lambda)=0$ should exist. I see none. – Did Oct 5 '11 at 6:58
Isn't $L(\lambda)$ continuous in $\lambda$? – Ashok Oct 5 '11 at 7:17
?? And so what? – Did Oct 5 '11 at 7:19
$L$ assumes positive value at $\lambda=0$ and negative at $\lambda=1$, so by intermediate value theorem, it must assume $0$ at some $\lambda$ in between, isn't it? Or am I lost completely? Sorry if I'm mistaken. – Ashok Oct 5 '11 at 7:29
Sure, this argument is already in your post and is correct and shows that $L(\lambda)=0$ for some (unique) $\lambda$. But this is not your question, is it? – Did Oct 5 '11 at 7:35

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.