# How do we know logistic loss is a non convex and log of logistic loss in convex?

I'm currently learning machine learning logistic regression and I'm really confused with logistic loss. I know the formula for logistic loss is $$\displaystyle g(z)=\frac{1}{1+e^{-z}}$$ here $$z = W^TX_i$$

But I've read that this is a non convex function. So we usually take logarithm of logistic loss to make it a convex function for optimization techniques like gradient descent etc..

$$J(\theta) =-\frac{1}{m}\sum_{i=1}^{m}y^{i}\log(h_\theta(x^{i}))+(1-y^{i})\log(1-h_\theta(x^{i}))$$ $$h_\theta(x^i) = \frac{1}{1+e^{-z}}$$

How can we know if a functions/loss is convex or non convex? Why are we taking log for logistic loss? How can that makes the loss function convex?

• "A differentiable function of one variable is convex on an interval if and only if its derivative is monotonically non-decreasing on that interval. If a function is differentiable and convex then it is also continuously differentiable"--en.wikipedia.org/wiki/Convex_function#Functions_of_one_variable – saulspatz Jul 8 '19 at 23:49

$$g$$ is not convex: consider the points $$0,1,N$$ where $$N$$ is a positive integer $$>1$$. We have $$1=\frac 1 N (N)+(1-\frac 1 N) 0$$. If $$g$$ is convex then we would have $$g(1) \leq \frac 1 N g(N)+(1-\frac 1 N)g(0)$$. If you let $$N \to \infty$$ in this you get the contradiction that $$e \leq 1$$.
$$\log g$$ is also not convex. It is concave. So $$-\log\, g$$ is convex. To see this write $$-\log\, g$$ as $$-\log \, \frac {e^{x}} {1+e^{x}}=-x +\log \, (1+e^{x})$$. it is easy to calculate the second derivative of the function and show that it is positive.
• The first derivative is $-1+\frac {e^{x}} {1+e^{x}}=-\frac 1 {1+e^{x}}$. Since the denominator is increasing the ratio is decreasing and the minus sign makes it increasing. – Kavi Rama Murthy Jul 9 '19 at 0:43
• No. To show that $g$ is not convex I did not use derivatives at all. I used just the definition of a convex function. But to show that $-\log\, g$ is convex I used derivatives. So the derivative I calculated are for $-\log\, g$. – Kavi Rama Murthy Jul 9 '19 at 5:31