# Show that logistic regression with squared loss function is non-convex

How would you show that if you do logistic regression with a squared loss function, it is not a convex optimization problem (in parameters)?

In other words, your loss function for an individual observation is $(y - p)^2$, where $y$ is the dependent variable and $p$ is your prediction, and $$p = \frac{1}{1+\exp(-w^Tx)},$$ where $x$ is the vector of predictors and $w$ is the vector of weights.

• You could show that the definition of a convex function is violated. It just takes one $w$, one $y$, two $x$ vectors, and a $\lambda$. – LinAlg Oct 25 '16 at 21:34

As long as $x$ is not zero, the squared error loss with respect to $w$ will be non-convex.
To follow up with Qiang Chen's answer, the red and orange curves are the squared loss functions in question, and they are both non-convex (not fully convex). They are also non-concave (not fully concave). They are half convex on one side and half concave on the other, with a middle inflection point where $f(x)=0$. Convex and concave functions are ones that always curve in the same direction, either up or down, they can not have that kind of "S" shape that both the red and orange curves have.
All the $x_1$ and $x_2$ values on the concave side of the function do not satisfy that definition of a convex function.