# How to show negative entropy function $f(x)=x\log(x)$ is strongly convex?

Let $$f: \mathbb{R}_+ \rightarrow \mathbb{R}$$ where $$f(x)=x\log(x)$$. How to show it is strongly convex, i.e.,

Definition: Let $$f: \mathbb{R}^n \rightarrow \mathbb{R}$$ be differentiable. Then $$f$$ is strongly convex if $$\exists$$ a positive constant $$\alpha > 0$$ such that $$\langle \nabla f(y) - \nabla f(x),y-x \rangle \geq \alpha||y-x||^2 \,\,\,\,\,,\,\,\,\forall x,y \in \mathbb{R}^n \tag{1}$$ Following $$(1)$$ we have $$(y-x)\log(\frac{y}{x})\geq \alpha(y-x)^2 \,\,\,\,\, \forall x,y \in \mathbb{R}_+$$.

What $$\alpha$$ satisfies the above and how we can handle the inequality $$\forall x,y \in \mathbb{R}_+$$ ?

• Why don't you calculate the second derivative? – Dr. Sonnhard Graubner Jan 17 at 17:53
• @Dr. Sonnhard Graubner : Because I want to better understand the definition to proof the general case:math.stackexchange.com/questions/3055187/… and solve it. – Saeed Jan 17 at 17:57
• @Dr. Sonnhard Graubner : Also, the second derivative does not specify $\alpha$. Actually, I do not know how to use this fact saying strongly convex implies $\nabla^2f(x) \succeq \alpha I$ for this case to get $\alpha$. – Saeed Jan 17 at 18:01
• The case $x=1,\,y=1+z>0$ gives $\frac{\ln(1+z)}{z}\ge\alpha$, which for sufficiently large $z>0$ contradicts any proposed $\alpha>0$. – J.G. Jan 17 at 18:27
• @J.G. : I think I should have assumed that $0 \leq x \leq 1$? – Saeed Jan 17 at 18:32

We have $$f'(x)=1+\log(x)$$. Strict convexity therefore means that there exists a strictly positive $$\alpha$$ such that $$[\log(y)-\log(x)](y-x)\geq\alpha(y-x)^2$$ holds. Without loss of generality I assume $$y>x$$. Then the above inequality requires that $$\log(y)-\log(x)\geq\alpha(y-x).$$ Although you don't state it, I assume that the variables $$x$$ and $$y$$ live on $$(0,1)$$ (because they are probabilities). Since the slope of the $$\log$$ function on the interval $$(0,1)$$ is larger than or equal to 1, you can choose any positive $$\alpha$$ smaller than 1.
If $$x$$ and $$y$$ live on a general bounded interval $$(0,M)$$, then the argument goes through with $$\alpha<1/M$$.
• I understand the argument justifying $\log(y)-\log(x)\geq\alpha(y-x)$ but could you show it algebraically? – Saeed Jan 17 at 18:53
• @Saeed: $\log$ is concave. Hence, $\log(x)\leq\log(y)+(1/y)(x-y)$. Since $1/y\geq1/M$, one gets $\log(y)-\log(x)\geq(1/y)(y-x)\geq(1/M)(y-x)$. – Gerhard S. Jan 17 at 19:27