# What is the intuition behind a function being 'weakly differentiable'?

As part of an optimization paper I am reading now, they are talking about a function $g$ being "weakly differentiable".

I looked it up on the wiki but I do not have enough context to start cracking into it.

I understand what it means for a function to be differentiable, (converges to the same answer if taken in the limit from both sides, IIRC), but what does it mean for a function to be weakly differentiable? What would such a function look like? What would it not look like?

Thanks!

-
This has to do with "generalized functions" or "distributions". If you have not heard of them, then our explanation of "weakly differentiable" probably won't help you much. –  GEdgar Jan 25 '13 at 3:32
@GEdgar This is why I put intuition in the title. I am not looking for an exposition, (if I was, wikipedia would have sufficed), but an intuitive explanation. One thing that would help are examples of functions that are weakly differentiable. (For example, y = |x| is mentioned here. That is great - what other functions might also be weakly differentiable?) Examples like this can go along way to color my understanding of the issue. Thanks! –  Mohammad Jan 25 '13 at 16:50

The basic intuition is that a weakly differentiable function looks differentiable except for on sets of zero measure. This allows functions that are not normally considered differentiable at "corners" to have a weak derivative that is defined everywhere on the original function's domain. The reason why weak derivatives ignore sets of zero measure is precisely because weak derivatives are defined by integrals, and integrals cannot see behavior on sets of zero measure.

The two examples on Wikipedia are great examples of what weak derivatives can look like:

1. $f(x) = |x|$. Classically, the derivative of $f$ is only defined when $x \neq 0$. However, the weak derivative ignores the behavior of this corner, and thus any function that equals $-1$ almost everywhere for $x < 0$, and $1$ for $x> 0$ is a weak derivative for $f(x)$. Notice that we can choose any value for the weak derivative at $x = 0$.

2. $f(x) = 1_{\mathbb{Q}}$, the characteristic function of the rationals (a.k.a. $f(x) = 1$ when $x$ is rational, $f(x) = 0$ when $x$ is irrational). What does this function look like? It is "mostly" zero, since there are uncountably many irrationals and only countably many rationals. Thus, intuitively, $f(x)$ is constant up to a set of zero measure. Notice, however, that normally this function is differentiable nowhere, but if were to just be able to modify this function on a set of rationals, we could make it differentiable. This intuition carries over to the actual weak derivative, which must be a function that is equal to zero almost everywhere.

Now, it is also important to recognize what a weak derivative cannot be, or more precisely, when a function does not have a weak derivative. Intuitively, functions that have discontinuities that cannot be "fixed" by changing the function on a set of zero measure cannot be weakly differentiable. Consider the function $$f(x) = \begin{cases} -1 & x \le 0 \\ 1 & x > 0 \end{cases}$$ I claim that $f(x)$ is not weakly differentiable. To prove this, take any $\varphi \in C_c^{\infty}(\mathbb{R})$. Then $$\int_{\mathbb{R}} \varphi'(x) f(x) \, dx = \int_{-\infty}^0 - \varphi'(x) \, dx + \int_{0}^{\infty} \varphi'(x) \, dx = -2 \varphi(0)$$ If $f(x)$ has a weak derivative $v(x)$, then it would follows that, for all $\varphi \in C_c^{\infty}(\mathbb{R})$, we would have the identity $$\int_{\mathbb{R}} \varphi(x) v(x) \, dx = 2 \varphi(0)$$ Now choose a sequence of compactly supported smooth bump functions $\varphi_n(x)$ such that $0 \le \varphi_n \le 1$, $\varphi(0) = 1$, and $\varphi_n(x) \rightarrow 0$ pointwise for each $x \neq 0$, as $n \rightarrow \infty$. Then $$2 = \lim_{n \rightarrow \infty} \int_{\mathbb{R}} \varphi_n(x) v(x) \, dx = 0,$$ a contradiction.

-
Thanks Christopher, what other examples of functions that are weakly differentiable come to mind? For example, what about the sign(x[n]) function? –  Mohammad Jan 25 '13 at 6:13
What do you mean by sign(x[n])? –  Christopher A. Wong Jan 25 '13 at 23:58
Christopher, by sign(x[n]) I just mean the non-linear operation of taking the sign of that particular number in the vector. For example, in my case, if my x[n] = [-2 1 4 -18 4 -3], then sign(x[n]) = [-1 1 1 -1 1 -1] –  Mohammad Jan 26 '13 at 0:03
@Mohammad: Your $\operatorname{sign}$ function has Christopher's $f(x) = \begin{cases} -1 & x \le 0 \\ 1 & x > 0 \end{cases}$ as a special case for $n=1$. It is not weakly differentiable. –  Rahul Jan 27 '13 at 18:27

I'll just give the idea for weak differentiability of functions $u: \Bbb R \longrightarrow \Bbb R$; the ideas extend to more general cases without too much effort.

If a function $u: \Bbb R \longrightarrow \Bbb R$ is differentiable, then by integration by parts, we know that for any differentiable function $\phi: \Bbb R \longrightarrow \Bbb R$ such that $\phi(x) = 0$ for $|x|$ large (i.e. $\phi$ is zero outside a bounded subset of the real line) the following holds: $$\int_{-\infty}^\infty u \phi' ~dx = \left. u\phi \right|_{-\infty}^\infty - \int_{-\infty}^\infty u' \phi ~dx = - \int_{-\infty}^\infty u' \phi ~dx.$$ The idea behind weakly differentiable functions is that we ask for this integration by parts formula to hold, but we don't assume that $u$ actually has a derivative in the normal sense.

To be precise, a function $u: \Bbb R \longrightarrow \Bbb R$ is weakly differentiable with weak derivative $v$ if there exists a function $v: \Bbb R \longrightarrow \Bbb R$ such that $$\int_{-\infty}^\infty u \phi' ~dx = - \int_{-\infty}^\infty v \phi ~dx$$ for all smooth functions $\phi: \Bbb R \longrightarrow \Bbb R$ that vanish outside some bounded set.

Functions which are not differentiable can still be weakly differentiable. For example, $$u(x) = |x|$$ is not differentiable because of the corner at $x = 0$, but it does have weak derivative $$v(x) = \begin{cases} -1, & x < 0, \\ 0, & x = 0, \\ 1, & x > 0, \end{cases}$$ as can easily be checked (note that we should consider integration here as Lebesgue integration).

Weakly differentiable functions can be very badly behaved. For example, one can construct a weakly differentiable function on the unit ball $B(0,1) \subset \Bbb R^n$ that is unbounded on every open subset of $B(0,1)$. On the other hand, functions with "too big" of a jump discontinuity, such as the weak derivative of $|x|$ above, are not weakly differentiable.

-
Thanks Henry, I see that both answers have the y = |x| as an example. What other examples of weakly differentiable functions might there be besides this? –  Mohammad Jan 25 '13 at 6:14
I guess my understanding of weakly differentiable is different... I can't think of a weakly differentiable function on a line that is not locally bounded. It seems the issue is: what do we require of $v$, and in what sense do we integrate $v\phi$? –  user53153 Jan 25 '13 at 6:39
@5PM: Let $B(0,1)$ be the open unit ball in $\Bbb R^n$, and let $\{q_k\}$ be an enumeration of the points with rational coordinates in $B(0,1)$. Define $$u(x) = \sum_{k=1}^\infty \frac{|x - q_k|^{-\alpha}}{2^k}.$$ Then $u \in W^{1,p}(B(0,1))$ for $0 < \alpha < \frac{n-p}{p}$, but $u$ is unbounded on any open subset of $B(0,1)$ for such $\alpha$. –  Henry T. Horton Jan 25 '13 at 16:44
@HenryT.Horton I wrote on a line, addressing your statement "one can construct a weakly differentiable function that is unbounded on every open interval." –  user53153 Jan 25 '13 at 17:05