We want to come up with a definition of functional derivative that allows us to harness the principle that extremal points are stationary points in order to solve problems involving optimising functionals.

Let us recapitulate how derivatives are generalised from single-real-variable to many-real-variables: let $f:\mathbf{R}^{d}\to\mathbf{R}$; one can define then

$$ \partial_{\vec{v}}f(\vec{x}) := \lim_{\varepsilon\to0} \frac{f(\vec{x}+\varepsilon\vec{v}) - f(\vec{x})}{\varepsilon} $$

to be the derivative of $f$ in the direction of $\vec{v}\in\mathbf{R}^{d}$ at the point $\vec{x}\in\mathbf{R}^{d}$.

I thought then of doing similarly for functionals $F:C(\mathbf{R})\to\mathbf{R}$: define

$$ \partial_{\eta} F[f] := \lim_{\varepsilon\to0} \frac{F[f+\varepsilon\eta] - F[f]}{\varepsilon} $$

to be the functional derivative of $F$ in the "direction" of $\eta\in C(\mathbf{R})$ at the "point" $f\in C(\mathbf{R})$ (addition and scalar multiplication of functions is done pointwise). Then, like we say that an $f:\mathbf{R}^{d}\to\mathbf{R}$ is stationary at a point $\vec{x}\in\mathbf{R}^{d}$ if

$$ \forall\vec{v}\in\mathbf{R}^{d}\qquad \partial_{\vec{v}}f(\vec{x}) = 0 $$

we can say that an $F:C(\mathbf{R})\to\mathbf{R}$ is stationary at a point $f\in C(\mathbf{R})$ if

$$ \forall\eta\in\mathbf C(\mathbf{R}) \qquad \partial_{\eta} F[f] = 0 $$

and then, if we wish to optimise some functional $F$ we turn to the functional equation $\partial_{\eta}F[f] = 0$.

It seems however, unfortunately, that the above definition of functional derivative is not useful. I tried using it to optimise certain functionals but it ended up in nonsense (see below). Additionally, according to Wikipedia the functional derivative is defined differently, albeit similarly; specifically: the functional derivative $\frac{\delta F}{\delta f}$ (seemingly a real function) is defined to satisfy

$$ \int \frac{\delta F}{\delta f}(x) \eta(x) \, dx = \partial_{\eta}F[f]$$

where the RHS employs the notation developed in this post.

I would be interested in justifications for this definition: why is it the correct one? And what is wrong with my definition?, why does it fail to be useful?

Here's an example of trying to use $\partial_{\eta}F = 0$ to optimise a functional $F$: let

$$ F[f] = \int\limits_{0}^{1}\sqrt{1 + (f'(x))^{2}} \, dx $$

we wish to minimise $F$ subject to $f(0) = a$ and $f(1) = b$, say. Then

\begin{align*} \partial_{\eta} F[f] &= \lim_{\varepsilon\to0}\frac{\int\limits_{0}^{1}\sqrt{1+(f(x)+\varepsilon\eta(x))'^{2}} \, dx- \int\limits_{0}^{1}\sqrt{1+f'(x)^{2}}\,dx}{\varepsilon} \\ &= \lim_{\varepsilon\to0} \int\limits_{0}^{1}\frac{\sqrt{1+(f'(x)+\varepsilon\eta'(x))^{2}} - \sqrt{1+f'(x)^{2}}}{\varepsilon}\, dx \\ &= \int\limits_{0}^{1}\lim_{\varepsilon\to0}\frac{\sqrt{1+(f'(x)+\varepsilon\eta'(x))^{2}} - \sqrt{1+f'(x)^{2}}}{\varepsilon}\, dx \\ &= \int\limits_{0}^{1}\frac{f'(x)\eta'(x)}{\sqrt{1+f'(x)^{2}}}\,dx = 0 \qquad \forall\eta \end{align*}

Now, by to the fundamental lemma of the calculus of variations,

\begin{align*} \frac{f'(x)}{\sqrt{1+f'(x)^{2}}} &= 0 \qquad \forall x\\ \therefore f'(x) &= 0 \qquad \forall x \end{align*}

but this is incorrect. The solution is a straight line, whose derivative is not necessarily identically zero. What am I doing wrong?


Your definition of $\partial_\eta F[f]$ is fine, and is perfectly analogous to the directional derivative in multivariable calculus.

From multivariable calculus, we know that for a differentiable function $g:\mathbb R^n \to \mathbb R$ we have $$ D_u g(x) = \langle \nabla g(x), u \rangle, $$ where $D_u g(x)$ is the directional derivative of $g$ at $x$ in the direction $u$. Well, the analogous statement when working with functionals is $$ \partial_{\eta}F[f] = \int \underbrace{\frac{\delta F}{\delta f}(x)}_{\text{analogous to gradient}} \eta(x) \, dx. $$ On the right, we are taking the inner product of the "gradient" of $F$ at $f$ and the function $\eta$.

So, these two notions of derivative exist in perfect harmony, just as in multivariable calculus.

| cite | improve this answer | |
  • $\begingroup$ Ah... yes! I understand it. Thank you. That settles the motivation for this definition part, but there's still one thing I am having trouble with: I don't seem to be able to use the equation $\partial_{\eta}F = 0$ to optimise a functional $F$. I'm gonna edit the OP to talk about this in greater depth... $\endgroup$ – what a disgrace Jun 13 '18 at 6:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.