10
$\begingroup$

I am trying to get the derivative of $|x|$, and I want that derivative function, say $g(x)$, to be a function of x.

So it really needs the |x| to be smooth (ex. $x^2$); I am wondering what is the best way to approximate |x| with something smooth?

I propose $\sqrt{x^2 + \epsilon}$, where $\epsilon = 10^{-10}$, but there should be something better? Perhaps Taylor expansion?

Sorry for any confusion. I should add some additional information here:

I want to use $|x|$ as part of an object function $J(x)$ which I want to minimize. So it would be nice to approximate $|x|$ with some smooth function so that I can get the analytic form of the first-order derivative of $J(x)$.

Thanks a lot.

$\endgroup$
  • 2
    $\begingroup$ @Thomas But "in $x = 0$ it is not defined" is exactly what Ono wants to fix by approximating $|x|$. That is what this question is about. It's not "how to differentiate $|x|$?", it's "How to 'fix' the non-differentiability of $|x|$ at $x = 0$ by approximation?" $\endgroup$ – Arthur Mar 26 '14 at 19:53
  • 1
    $\begingroup$ @Thomas Yes, it can. You "smooth out" the corner. The function $\sqrt{x^2 + \epsilon}$ is one way to do just that. Another way would be to swap a small portion of the graph around $0$ with a quarter circle, but that's not nearly as smooth if you do it naively. You can do a patching like that smoothly, though, if you're smart about it, so that everywhere outside a small neighbourhood of $0$ the functions are exactly the same, and the patched function is everywhere infinitely differentiable. $\endgroup$ – Arthur Mar 26 '14 at 19:58
  • 1
    $\begingroup$ @Arthur The question is about (literal citation) "a derivative of $|x|$". This does not exist. It is out of question that you can approximate continuous functions, say, uniformly, by smooth functions. If that is what is asked for, the question should specify that. It does not. $\endgroup$ – Thomas Mar 26 '14 at 20:02
  • 2
    $\begingroup$ In fairness, both the title and the statement 'I am wondering what is the best way to approximate |x| with something smooth?' suggest a little more than an attempt to compute the derivative. $\endgroup$ – copper.hat Mar 26 '14 at 20:07
  • 2
    $\begingroup$ @Thomas Have you read the title to the question? Titles usually give some indication as to what is asked about. I cite litterally: "Approximate $|x|$ with a smooth function". That can be done. Let $f(x)$ be a smooth function that is zero everywhere outside some small interval $(\epsilon, \epsilon)$, tops out at $f(x) = 1$ in some even smaller interval around $1$, and monotone in between. Then $(1-f(x))|x| + f(x)h(x)$ is smooth as long as $h(x)$ is smooth inside $(-\epsilon, \epsilon)$, and it's a good approximation if $h(x)$ is say, a quarter circle or parabola "glued" to the bottom of $|x|$. $\endgroup$ – Arthur Mar 26 '14 at 20:09
6
$\begingroup$

I have used your $\sqrt{x^2+\epsilon}$ function once before, when an application I was working on called for a curve with a tight radius of curvature near $x=0$. Whether it is best for your purpose might depend on your purpose.

(Side note: The derivative of $|x|$ does not exist at $x=0$. In physics, we sometimes cheat and write ${d\over dx}|x| = \rm{sgn}(x)$, the "sign" function that gives $-1$ when $x<0$, $0$ when $x=0$, and $+1$ when $x > 0$. But only when we know that the value at $x=0$ will not get us into trouble!)

$\endgroup$
8
$\begingroup$

A bit late to the party, but you can smoothly approximate $f(x) = |x|$ by observing $\partial f/\partial x = \mbox{sgn}(x)$ for $x \neq 0$. Therefore approximating the $\mbox{sgn}(x)$ function by

$$ f(x) = 2\mbox{ sigmoid}(kx)-1$$

(the $k$ being a parameter that allows you to control the smoothness), we get

$$ \partial f/\partial x = 2\left (\frac{e^{kx}}{1+e^{kx}} \right ) - 1 $$ $$ \Rightarrow f(x) = \frac{2}{k}\log(1+e^{kx})-x-\frac{2}{k}\log(2)$$

where the constant term was chosen to ensure $f(0) = 0 $.

Included are plots of the function for $x \in [-5,5]$, where $k=1$ is red, $k=10$ is blue and $k=100$ is black.

Note that if you wanted to use the more smooth $k=1$ in the interval $[-5,5]$ it may be worth applying a further linear transformation i.e. ~$5f(x)/3.6$ to ensure the values at the edge of the interval are correct.

enter image description here

$\endgroup$
1
$\begingroup$

Another solution would be the function $x \rightarrow x \frac{e^{kx}}{e^{kx} + e^{-kx}} -x\frac{e^{-kx}}{e^{kx} + e^{-kx}} $ with higher the $k$, closer you will be to the absolute value function. Furthermore it is equal to 0 in 0 wich is sometimes a desirable feature (compared to the solution with the root square stated above). However you lose the convexity.

$\endgroup$
0
$\begingroup$

$$\forall x\neq0,\dfrac{d|x|}{dx} = \frac{x}{|x|} = \begin{cases} -1 & x<0 \\ 1 & x>0. \end{cases}$$

$\endgroup$
  • 1
    $\begingroup$ This is not what @Ono is after. $\endgroup$ – Arthur Mar 26 '14 at 19:53
  • $\begingroup$ Upvote, simply because I think the reason for the downvote is nonsense. This is the derivative. $\endgroup$ – Thomas Mar 26 '14 at 19:58
  • 1
    $\begingroup$ @Arthur The OP said that he wanted the derivative of $|x|$ to be a function of $x$. That is what I did $\endgroup$ – user122283 Mar 26 '14 at 19:59
  • $\begingroup$ @SanathDevalapurkar The OP also states that by approximating $|x|$ she wants a smooth solution. That is what the question is about. I.e. what smooth approximations to the absolute value function are useful. $\endgroup$ – Arthur Mar 26 '14 at 20:00
  • $\begingroup$ FWIW, I think a good answer should at least refer to OP's proposal, since OP asks whether the proposal is best. $\endgroup$ – Jason Zimba Mar 26 '14 at 20:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.