I've been reading lately about reproducing kernel Hilbert spaces (RKHS) and Gaussian processes (GP) and during my studies I came across with the concept of weak derivative and Sobolev spaces. I have tried to get some sense from weak derivative by reading these two questions at the site:
What is the intuition behind a function being 'weakly differentiable'?
Intuition about weakly differentiable functions
So I have learned now that:
A function $u:\Omega\rightarrow\mathbb{R}$ is weakly differentiable with $\alpha$th ($\alpha\in\mathbb{N}^n$) weak partial derivative $v=D^{\alpha}u$, if for all compactly supported smooth functions $\phi$ it holds that:
$$\int_{\Omega} u \,D^{\alpha}\phi\,dx = (-1)^{|\alpha|}\int_{\Omega}v\phi\,dx.$$
Quoting earlier answers (by user Christopher A. Wong) from the links I provided above:
The basic intuition is that a weakly differentiable function looks differentiable except for on sets of zero measure. This allows functions that are not normally considered differentiable at "corners" to have a weak derivative that is defined everywhere on the original function's domain. The reason why weak derivatives ignore sets of zero measure is precisely because weak derivatives are defined by integrals, and integrals cannot see behavior on sets of zero measure.
Now I'm almost satisfied with this answer, but one unclarity remains:
Why is the weak derivative defined as it is? That is, when the idea of weak derivative was first considered, whoever it was, why did he select that particular equation above as the definition? Or was this definition a somewhat arbitrary choice, which simply suited to our purposes?
For example, one of the reasons of using squared error in many problems of statistics or optimization is (as I've understood) because the squared error has nice analytical properties, making the math easy, when compared with e.g. absolute error.
The reason why I'm asking this, is that sometimes it seems many techniques of mathematics rise like from a magicians hat. For a beginner like myself, it is difficult to picture the scenario and conditions that gave rise to that discovery or definition.