Let $f: \mathbb{R}_+ \rightarrow \mathbb{R}$ where $f(x)=x\log(x)$. How to show it is strongly convex, i.e.,

Definition: Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ be differentiable. Then $f$ is strongly convex if $\exists$ a positive constant $\alpha > 0$ such that $$ \langle \nabla f(y) - \nabla f(x),y-x \rangle \geq \alpha||y-x||^2 \,\,\,\,\,,\,\,\,\forall x,y \in \mathbb{R}^n \tag{1} $$ Following $(1)$ we have $(y-x)\log(\frac{y}{x})\geq \alpha(y-x)^2 \,\,\,\,\, \forall x,y \in \mathbb{R}_+$.

What $\alpha$ satisfies the above and how we can handle the inequality $\forall x,y \in \mathbb{R}_+$ ?

  • $\begingroup$ Why don't you calculate the second derivative? $\endgroup$ – Dr. Sonnhard Graubner Jan 17 at 17:53
  • $\begingroup$ @Dr. Sonnhard Graubner : Because I want to better understand the definition to proof the general case:math.stackexchange.com/questions/3055187/… and solve it. $\endgroup$ – Saeed Jan 17 at 17:57
  • $\begingroup$ @Dr. Sonnhard Graubner : Also, the second derivative does not specify $\alpha$. Actually, I do not know how to use this fact saying strongly convex implies $\nabla^2f(x) \succeq \alpha I$ for this case to get $\alpha$. $\endgroup$ – Saeed Jan 17 at 18:01
  • $\begingroup$ The case $x=1,\,y=1+z>0$ gives $\frac{\ln(1+z)}{z}\ge\alpha$, which for sufficiently large $z>0$ contradicts any proposed $\alpha>0$. $\endgroup$ – J.G. Jan 17 at 18:27
  • $\begingroup$ @J.G. : I think I should have assumed that $0 \leq x \leq 1$? $\endgroup$ – Saeed Jan 17 at 18:32

We have $f'(x)=1+\log(x)$. Strict convexity therefore means that there exists a strictly positive $\alpha$ such that $$[\log(y)-\log(x)](y-x)\geq\alpha(y-x)^2$$ holds. Without loss of generality I assume $y>x$. Then the above inequality requires that $$\log(y)-\log(x)\geq\alpha(y-x).$$ Although you don't state it, I assume that the variables $x$ and $y$ live on $(0,1)$ (because they are probabilities). Since the slope of the $\log$ function on the interval $(0,1)$ is larger than or equal to 1, you can choose any positive $\alpha$ smaller than 1.

If $x$ and $y$ live on a general bounded interval $(0,M)$, then the argument goes through with $\alpha<1/M$.

  • $\begingroup$ I understand the argument justifying $\log(y)-\log(x)\geq\alpha(y-x)$ but could you show it algebraically? $\endgroup$ – Saeed Jan 17 at 18:53
  • $\begingroup$ @Saeed: $\log$ is concave. Hence, $\log(x)\leq\log(y)+(1/y)(x-y)$. Since $1/y\geq1/M$, one gets $\log(y)-\log(x)\geq(1/y)(y-x)\geq(1/M)(y-x)$. $\endgroup$ – Gerhard S. Jan 17 at 19:27
  • $\begingroup$ Smart way of showing that. $\endgroup$ – Saeed Jan 17 at 19:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.