Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

is there a way to express the logistic function $$\frac{1}{1+\exp(-x)}$$ as the difference of two convex functions?

Thanks

share|improve this question
    
Can you provide some background for this question? Although your question is a good question, there is an aversion to questions in which no effort is shown. If you provide some background or, if this is a homework question, your work so far, your question may not be closed, or if it is closed, it may be able to be reopened. –  robjohn Feb 24 at 10:49
    
To those who have voted to close: please let the author know why you have voted to close and what they can do to improve their question. If they are new to the site, they may not know about the close votes. –  robjohn Feb 24 at 10:52

3 Answers 3

up vote 0 down vote accepted

A convex function has an increasing derivative and we can write any function of bounded variation as a difference of two increasing functions. So let us look at the derivative $$ \begin{align} \frac{\mathrm{d}}{\mathrm{d}x}\frac1{1+e^{-x}} &=\frac{e^{-x}}{(1+e^{-x})^2}\\ &=\frac1{(e^{x/2}+e^{-x/2})^2}\\ &=\tfrac14\,\mathrm{sech}^2(x/2) \end{align} $$ Thus, we can break this up into the difference of two increasing functions: $$ f'(x)=\left\{\begin{array}{l} \tfrac14&\text{when }x\ge0\\ \tfrac14\,\mathrm{sech}^2(x/2)\hphantom{-4}&\text{when }x\lt0 \end{array}\right. $$ and $$ g'(x)=\left\{\begin{array}{l} \tfrac14-\tfrac14\,\mathrm{sech}^2(x/2)&\text{when }x\ge0\\ 0&\text{when }x\lt0 \end{array}\right. $$ Then if $$ f(x)=\left\{\begin{array}{l} \tfrac14x+\tfrac12&\text{when }x\ge0\\ \tfrac12\tanh(x/2)+\tfrac12&\text{when }x\lt0 \end{array}\right. $$ and $$ g(x)=\left\{\begin{array}{l} \tfrac14x-\tfrac12\tanh(x/2)&\text{when }x\ge0\\ 0&\text{when }x\lt0 \end{array}\right. $$ $f$ and $g$ are convex and $f(x)-g(x)=\frac12+\frac12\tanh(x/2)=\dfrac1{1+e^{-x}}$

share|improve this answer

$$ \frac{d^2}{dx^2} (1+e^x)^{-1} = \frac{d}{dx} \left(-(1+e^x)^{-2}e^x\right) = e^x\left( -(1+e^x)^{-2}+2(1+e^x)^{-3}e^x \right) $$ $$ = \frac{e^x (-(1+e^x)+2e^x)}{(1+e^x)^3} = \frac{ e^x(e^x-1) }{(1+e^x)^3}. $$ This is a bounded function because it is everywhere continuous and goes to $0$ as $x\to\pm\infty$.

So let $f(x) = Ax^2 + \dfrac{1}{1+e^x}$ with $A$ big enough so that the second derivative $f''$ is always positive. Then the logistic function is the difference between the convex function $f$ and the convex function $x\mapsto Ax^2$.

share|improve this answer
    
Thanks, but I meant if there was a "fundamental" way. Let me explain. For instance, consider $$g(x) \triangleq \frac{1}{t} (x)^+$$, where $t>0$ and $(\cdot)^+$ denotes the positive part. We have something similar to the logistic function by $g(x+\frac{t}{2})-g(x-\frac{t}{2})$. Is there a similar way to write the logistic (perhaps as difference of two exponential functions, just guessing)? Thanks –  Ita Atz Feb 23 at 19:29
    
This certainly answers the question. Did you compute the minimum of $A$? –  robjohn Feb 24 at 16:46

A different approach.

If $f$ is any function with continuous second derivative let $$ f''_+=\max(f'',0)\quad f''_-=-\min(f'',0). $$ Then $f''_+$ and $f''_-$ are continuous, non-negative and $f''=f''_+-f''_-$. Now let $F_+$ and $F_-$ be such that $F_+''=f''_+$ and $F_-''=f''_-$. $F_+$ and $F_-$ are convex, and the constants of integration can be chosen so that $f=F_+-F_-$.

This is analogous to writing any $C^1$ function as the difference of two increasing functions.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.