# Negative Entropy $x\log x$ is convex

can someone tell me (or show me), where can I find a proof for $f:\mathbb{R}_{\geq0}\to\mathbb{R},\quad x\mapsto x\log x$ is convex without using the first or second derivative trick?

We call a map convex, then $f(\lambda x+(1-\lambda)\tilde{x})\leq\lambda f(x)+(1-\lambda)f(\tilde{x})\quad\text{for}~\lambda\in(0,1)$

• I know, that the function is convex and I also know that the reason is that the seond derivative is positiv for all $x>0$. What I search is a proof without the second derivative argument. – FuncAna09 Jan 26 at 15:36
• Welcome @FuncAna09! Could you write the definition of convex function you know? It would be easier to help you. :) – Ixion Jan 26 at 15:38
• Why are you searching for a proof that avoids the second derivative? In this case, it's by far the cleanest argument. – Misha Lavrov Jan 26 at 15:51
• Simply because I want to know how the evidence would go without the derivation. – FuncAna09 Jan 26 at 15:58
• I think it might be helpful to consider a slight perturbation of the subtitle of Stanley Kubrick's movie "Doctor Strangelove": "How I learned to stop worrying and love the second derivative trick". – Lee Mosher Jan 26 at 16:52

## 3 Answers

Answer to previous version of this question:

This is hair-splitting, but: the first derivative of your function is an increasing function, so your function is convex.

Hair splitting, because the first derivative trick is not the second derivative trick, and "increasing first derivative" is not exactly "non-negative second derivative", but yet implies convexity, too.

• Okay that was my mistake. The question is as follows: Is there any evidence that only goes with the definition? – FuncAna09 Jan 26 at 16:38

So I tried the proof, to prove that a function or set is convex we take two points $$x$$ and $$y$$ and we try proving that , $$f(\lambda x + (1-\lambda) y) \leq \lambda f(x) + (1-\lambda) f(y)$$ The concept is simple if we joing two points $$x$$ and $$y$$ by a line then all the points lying on the line should also lie inside the set, the line being $$\lambda x + (1-\lambda) y$$ , if we vary \lambda from $$(0,1)$$ we find at $$\lambda$$ = $$0$$ we get the point $$x$$ at $$\lambda = 1$$ we get the point $$y$$ and for the value of $$\lambda = 0.5$$ we get the midpoint of $$x$$ and $$y$$.

Now for the proof: $$f(x) = x \log x$$ so $$f(\lambda x + (1-\lambda) y) = ({\lambda x + (1-\lambda) y})\space \log ({\lambda x +(1-\lambda)y})$$

The RHS becomes $$\lambda x \log (\lambda x + (1-\lambda) y)+ (1-\lambda) y \log (\lambda x + (1-\lambda) y$$

Take the first part and you will notice $$\lambda x \log(\lambda x + (1-\lambda)y) \geq \lambda x \log(\lambda x)$$ since $$\log(A+B) \geq \log A$$

Again $$\lambda x \log \lambda x = \lambda x (\log \lambda + \log x) \geq \lambda x \log x$$ So this leads to $$\lambda x \log(\lambda x + (1-\lambda)y) \geq \lambda x \log x$$ Similarly, $$(1-\lambda) y \log(\lambda x + (1-\lambda)y) \geq (1-\lambda) y \log y$$

Adding these two equations you get, **$$\lambda x \log(\lambda x + (1-\lambda)y) + (1-\lambda) y \log(\lambda x + (1-\lambda)y) \geq \lambda x \log x + (1-\lambda) y \log y$$

This proves $$f(\lambda x + (1-\lambda) y) \geq \lambda f(x) + (1-\lambda) f(y)$$ meaning that $$f(x) = xlogx$$ is not convex meaning that $$-f(x)=-x\log x$$ is convex.

Hope this helps …….

• This can not be, because the second derivative is $f''(x)=\frac{1}{x}>0~\forall x>0$. – FuncAna09 Jan 26 at 18:14
• $\log(\lambda x)<\log(x)$ because $0<\lambda<1$. This was your mistake. – FuncAna09 Jan 26 at 18:28

That's what I figured out: Let $$\lambda\in (0,1)$$ and $$x,y\in\mathbb{R}>0$$. Then we get \begin{align*} f(\lambda x+(1-\lambda)y)&=[\lambda x+(1-\lambda)y]\log(\lambda x+(1-\lambda)y)\\ \\ &=\lambda x\log(\lambda x+(1-\lambda)y)+(1-\lambda)y\log(\lambda x+(1-\lambda)y). \end{align*} For the first term of RHS we get $\lambda x\log(\lambda x+(1-\lambda)y)\leq\lambda x\log(x)+\lambda(1-\lambda)y$ and for the second term we get $(1-\lambda)y\log(\lambda x+(1-\lambda)y)\leq (1-\lambda)y\log(y)+\lambda(1-\lambda)x$ In summary, this results in $f(\lambda x+(1-\lambda)y)\leq \lambda f(x)+(1-\lambda)f(y)+\lambda(1-\lambda)(x+y).$ The last term is the problem now.