18
$\begingroup$

As part of an optimization paper I am reading now, they are talking about a function $g$ being "weakly differentiable".

I looked it up on the wiki but I do not have enough context to start cracking into it.

I understand what it means for a function to be differentiable, (converges to the same answer if taken in the limit from both sides, IIRC), but what does it mean for a function to be weakly differentiable? What would such a function look like? What would it not look like?

Thanks!

$\endgroup$
2
  • 2
    $\begingroup$ This has to do with "generalized functions" or "distributions". If you have not heard of them, then our explanation of "weakly differentiable" probably won't help you much. $\endgroup$
    – GEdgar
    Jan 25, 2013 at 3:32
  • 2
    $\begingroup$ @GEdgar This is why I put intuition in the title. I am not looking for an exposition, (if I was, wikipedia would have sufficed), but an intuitive explanation. One thing that would help are examples of functions that are weakly differentiable. (For example, y = |x| is mentioned here. That is great - what other functions might also be weakly differentiable?) Examples like this can go along way to color my understanding of the issue. Thanks! $\endgroup$
    – Spacey
    Jan 25, 2013 at 16:50

2 Answers 2

36
$\begingroup$

The basic intuition is that a weakly differentiable function looks differentiable except for on sets of zero measure. This allows functions that are not normally considered differentiable at "corners" to have a weak derivative that is defined everywhere on the original function's domain. The reason why weak derivatives ignore sets of zero measure is precisely because weak derivatives are defined by integrals, and integrals cannot see behavior on sets of zero measure.

The two examples on Wikipedia are great examples of what weak derivatives can look like:

  1. $f(x) = |x|$. Classically, the derivative of $f$ is only defined when $x \neq 0$. However, the weak derivative ignores the behavior of this corner, and thus any function that equals $-1$ almost everywhere for $x < 0$, and $1$ for $x> 0$ is a weak derivative for $f(x)$. Notice that we can choose any value for the weak derivative at $x = 0$.

  2. $f(x) = 1_{\mathbb{Q}}$, the characteristic function of the rationals (a.k.a. $f(x) = 1$ when $x$ is rational, $f(x) = 0$ when $x$ is irrational). What does this function look like? It is "mostly" zero, since there are uncountably many irrationals and only countably many rationals. Thus, intuitively, $f(x)$ is constant up to a set of zero measure. Notice, however, that normally this function is differentiable nowhere, but if were to just be able to modify this function on a set of rationals, we could make it differentiable. This intuition carries over to the actual weak derivative, which must be a function that is equal to zero almost everywhere.

Now, it is also important to recognize what a weak derivative cannot be, or more precisely, when a function does not have a weak derivative. Intuitively, functions that have discontinuities that cannot be "fixed" by changing the function on a set of zero measure cannot be weakly differentiable. Consider the function $$ f(x) = \begin{cases} -1 & x \le 0 \\ 1 & x > 0 \end{cases} $$ I claim that $f(x)$ is not weakly differentiable. To prove this, take any $\varphi \in C_c^{\infty}(\mathbb{R})$. Then $$ \int_{\mathbb{R}} \varphi'(x) f(x) \, dx = \int_{-\infty}^0 - \varphi'(x) \, dx + \int_{0}^{\infty} \varphi'(x) \, dx = -2 \varphi(0)$$ If $f(x)$ has a weak derivative $v(x)$, then it would follows that, for all $\varphi \in C_c^{\infty}(\mathbb{R})$, we would have the identity $$ \int_{\mathbb{R}} \varphi(x) v(x) \, dx = 2 \varphi(0) $$ Now choose a sequence of compactly supported smooth bump functions $\varphi_n(x)$ such that $0 \le \varphi_n \le 1$, $\varphi(0) = 1$, and $\varphi_n(x) \rightarrow 0$ pointwise for each $x \neq 0$, as $n \rightarrow \infty$. Then $$ 2 = \lim_{n \rightarrow \infty} \int_{\mathbb{R}} \varphi_n(x) v(x) \, dx = 0,$$ a contradiction.

$\endgroup$
5
  • $\begingroup$ Thanks Christopher, what other examples of functions that are weakly differentiable come to mind? For example, what about the sign(x[n]) function? $\endgroup$
    – Spacey
    Jan 25, 2013 at 6:13
  • $\begingroup$ What do you mean by sign(x[n])? $\endgroup$ Jan 25, 2013 at 23:58
  • $\begingroup$ Christopher, by sign(x[n]) I just mean the non-linear operation of taking the sign of that particular number in the vector. For example, in my case, if my x[n] = [-2 1 4 -18 4 -3], then sign(x[n]) = [-1 1 1 -1 1 -1] $\endgroup$
    – Spacey
    Jan 26, 2013 at 0:03
  • $\begingroup$ @Mohammad: Your $\operatorname{sign}$ function has Christopher's $f(x) = \begin{cases} -1 & x \le 0 \\ 1 & x > 0 \end{cases}$ as a special case for $n=1$. It is not weakly differentiable. $\endgroup$
    – user856
    Jan 27, 2013 at 18:27
  • 1
    $\begingroup$ @ChristopherA.Wong should that be let "$\varphi_n(0) = 1$"? $\endgroup$ Mar 24, 2017 at 13:11
8
$\begingroup$

I'll just give the idea for weak differentiability of functions $u: \Bbb R \longrightarrow \Bbb R$; the ideas extend to more general cases without too much effort.

If a function $u: \Bbb R \longrightarrow \Bbb R$ is differentiable, then by integration by parts, we know that for any differentiable function $\phi: \Bbb R \longrightarrow \Bbb R$ such that $\phi(x) = 0$ for $|x|$ large (i.e. $\phi$ is zero outside a bounded subset of the real line) the following holds: $$\int_{-\infty}^\infty u \phi' ~dx = \left. u\phi \right|_{-\infty}^\infty - \int_{-\infty}^\infty u' \phi ~dx = - \int_{-\infty}^\infty u' \phi ~dx.$$ The idea behind weakly differentiable functions is that we ask for this integration by parts formula to hold, but we don't assume that $u$ actually has a derivative in the normal sense.

To be precise, a function $u: \Bbb R \longrightarrow \Bbb R$ is weakly differentiable with weak derivative $v$ if there exists a function $v: \Bbb R \longrightarrow \Bbb R$ such that $$\int_{-\infty}^\infty u \phi' ~dx = - \int_{-\infty}^\infty v \phi ~dx$$ for all smooth functions $\phi: \Bbb R \longrightarrow \Bbb R$ that vanish outside some bounded set.

Functions which are not differentiable can still be weakly differentiable. For example, $$u(x) = |x|$$ is not differentiable because of the corner at $x = 0$, but it does have weak derivative $$v(x) = \begin{cases} -1, & x < 0, \\ 0, & x = 0, \\ 1, & x > 0, \end{cases}$$ as can easily be checked (note that we should consider integration here as Lebesgue integration).

Weakly differentiable functions can be very badly behaved. For example, one can construct a weakly differentiable function on the unit ball $B(0,1) \subset \Bbb R^n$ that is unbounded on every open subset of $B(0,1)$. On the other hand, functions with "too big" of a jump discontinuity, such as the weak derivative of $|x|$ above, are not weakly differentiable.

$\endgroup$
5
  • 1
    $\begingroup$ Thanks Henry, I see that both answers have the y = |x| as an example. What other examples of weakly differentiable functions might there be besides this? $\endgroup$
    – Spacey
    Jan 25, 2013 at 6:14
  • $\begingroup$ I guess my understanding of weakly differentiable is different... I can't think of a weakly differentiable function on a line that is not locally bounded. It seems the issue is: what do we require of $v$, and in what sense do we integrate $v\phi$? $\endgroup$
    – user53153
    Jan 25, 2013 at 6:39
  • $\begingroup$ @5PM: Let $B(0,1)$ be the open unit ball in $\Bbb R^n$, and let $\{q_k\}$ be an enumeration of the points with rational coordinates in $B(0,1)$. Define $$u(x) = \sum_{k=1}^\infty \frac{|x - q_k|^{-\alpha}}{2^k}.$$ Then $u \in W^{1,p}(B(0,1))$ for $0 < \alpha < \frac{n-p}{p}$, but $u$ is unbounded on any open subset of $B(0,1)$ for such $\alpha$. $\endgroup$ Jan 25, 2013 at 16:44
  • $\begingroup$ @HenryT.Horton I wrote on a line, addressing your statement "one can construct a weakly differentiable function that is unbounded on every open interval." $\endgroup$
    – user53153
    Jan 25, 2013 at 17:05
  • 2
    $\begingroup$ @HenryT.Horton You say "too big of a jump discontinuity"...does this mean there exist functions with a jump discontinuity that are "not too big" and hence are weakly differentiable? I thought a function had to be continuous, such as the absolute value your in your example, to be weakly differentiable? $\endgroup$ Mar 24, 2017 at 13:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .