# Functional Derivative

I am completely confused about calculating the (infinitesimal) change of an expression that depends on a function, when I vary the function. Consider the following: $$U(x;\rho)=\int_{0}^{x}\left[1-\int_{s}^{1}F(\rho(\xi))f(\xi)d\xi\right]^{n-1}ds$$ where $F:[0,1]\rightarrow [0,1]$ is continuous and increasing; $f(x)=\frac{dF(x)}{dx}$, $x\in[0,1]$, and $n>2$. Fix the value of $x$. Then, the above expression becomes a functional (?). Now, I want to compute the change in $U$ for a fix $x$ when the function $\rho$ varies. According to me, this is what the resulting expression should look like:

$$\frac{dU(\rho)}{d\rho}=(n-1)\int_{0}^{x}\left[1-\int_{s}^{1}F(\rho(\xi))f(\xi)d\xi\right]^{n-2}\left(\int_{s}^{1}f(\rho(\xi))f(\xi)d\xi\right)ds$$

Can someone please tell me if what I'm doing is correct? My confusion stems from the fact that for a given value of $x$, the above expression becomes a functional and I am not really confident about differentiating functionals.

Any help is really, really appreciated!!!

-
I think you are missing a minus sign, and also, you probably need $F$ to be differentiable in some uniform sort of way. You also need to specify some topology/norm for $\rho$. –  copper.hat Jul 6 '12 at 23:48
Thanks cooper.hat! Yup, I forgot a minus sign. What do I need F to be uniformly differentiable? For the norm I use the sup norm. Thanks again...and still waiting for more help! –  Cristian Jul 7 '12 at 0:10
I added more elaboration than you wanted to know below... –  copper.hat Jul 7 '12 at 5:52

Obtaining the Fréchet derivative is straightforward, basically the same as 'ordinary' differentiation, except that more attention needs to be paid to the details. You have a slight omission in your formula above; while notationally bothersome, I think it is a good idea to remember that the derivative is a linear operator, and to actually apply the derivative to some point. So, rather than computing $Df(x)$, instead compute $Df(x)(h)$. In $\mathbb{R}^n$, this is of little consequence, but in 'function' spaces it matters.

To informally guess the derivative, I use the 'little o' notation and then check the details. However, for the details. I think it is easiest to use composition rule, ie, if $f,g$ are (Fréchet) differentiable with appropriate domains/ranges, then $D f \circ g (x) = D f(g(x)) Dg(x)$.

You didn't mention what space $\rho$ 'lives in', I will take it to be $X=L_{\infty}[0,1]$. Also, I am going to assume that $F$ is continuously differentiable on $[0,1]$.

Define the following functions: $$\phi_1:X \to X, \ \ \phi_1(g)(t) = F(g(t))F'(t), \\ \phi_2:X \to X, \ \ \phi_2(g)(t) = \int_t^1 g(\tau) d\tau, \\ \phi_3:X \to X, \ \ \phi_3(g)(t) = (1-g(t))^{n-1}, \\ \phi_4:X \to \mathbb{R}, \ \ \phi_4(g)(t) = \int_0^x g(\tau) d\tau.$$ Then we have $U(x,\rho) = \phi_4(\phi_3(\phi_2(\phi_1(\rho))))$. I need to show that each is differentiable, and compute the derivative.

First, I need an estimate: Suppose $g:\mathbb{R} \to \mathbb{R}$ is continuously differentiable. Let $K \subset \mathbb{R}$ be compact, and $x \in K$. Then, for any $\delta$, Taylor's theorem gives: $$g(x+\delta)-g(x)-g'(x)\delta = \int_0^1 (g'(x+t \delta)-g'(x))\, dt \, \delta.$$ Now, since $g'$ is continuous, it is uniformly continuous on any compact set. It is straightforward to show that the set $K_{\alpha} = \{ x+y | x \in K, |y| \leq \alpha \}$ is compact, for any $\alpha$. Now let $\epsilon > 0$, and choose $\alpha > 0$ small enough so that $|g'(x)-g'(y)| < \epsilon$ whenever $x,y \in K_{\alpha}$, and $|x-y| < \alpha$. Then, if $x \in K$, and $|\delta| < \alpha$, we have $$|g(x+\delta)-g(x)-g'(x)\delta| < \epsilon |\delta|.$$

Consider $\phi_1$. Using the estimate, let $\epsilon >0$, and $K$ be a compact set. Then $\exists \alpha$ such that if $x \in K$, and $|\delta| < \alpha$, we have $|F(x+\delta)-F(x)-F'(x)\delta| < \epsilon |\delta|$. Also, note that $B=\sup_{t\in[0,1]} | F'(t)| < \infty$. Now suppose $g \in X$, then we have $|g(t)| \leq ||g||$ a.e. The set $K = \{x | |x| \leq ||g||\}$ is compact, so with $||\delta|| < \alpha$, we have the estimate: $$|F(g(t)+\delta(t))F'(t)-F(g(t))F'(t)-F'(g(t))F'(t)\delta(t)| < \epsilon B ||\delta||, \mbox{a.e.} \ t.$$ It follows (after checking for continuity, of course) that $D \phi_1(g)(\delta) (t) = F'(g(t))F'(t)\delta(t)$.

Thankfully $\phi_2$ is a little easier, it should be fairly clear that $D \phi_2(g)(\delta) (t) = \int_t^1 \delta(\tau) d \tau$.

$\phi_3$ follows the same lines as $\phi_1$, except applied to the function $x \mapsto (1-x)^{n-1}$. This yields $\phi_3(g)(\delta) (t) =-(n-1)(1-g(t))^{n-2} \delta(t)$.

Finally, $D \phi_4(g)(\delta) = \int_0^x \delta(\tau) d \tau$.

Now 'all' we need to do is compose the derivatives. We have $$D \phi_2 \circ \phi_1 (\rho) (\delta) (t) = D \phi_2 ( \phi_1 (\rho) ) D \phi_1 ( \rho) (\delta) (t) = \int_t^1 F'(\rho(\tau))F'(\tau) \delta(\tau) d \tau.$$ Next $$D \phi_3 \circ \phi_2 \circ \phi_1 (\rho) (\delta) (t) = -(n-1)(1-\int_t^1 F(\rho(\tau))F'(\tau) d \tau)^{n-2} \int_t^1 F'(\rho(\tau))F'(\tau) \delta(\tau) d \tau.$$

And finally (really), we have $$D \phi_4 \circ \phi_3 \circ \phi_2 \circ \phi_1 (\rho) (\delta) = -(n-1) \int_o^x (1-\int_t^1 F(\rho(\tau))F'(\tau) d \tau)^{n-2} \int_t^1 F'(\rho(\tau))F'(\tau) \delta(\tau) d \tau d t.$$

Notice how the 'perturbation' ($\delta$) works its way into the last integral. This was missing in your formula above.

-
What I nice, detailed and complete answer. I truly appreciate your time devoted to answer my question. It is pretty clear to me know how the functional derivative works now. The composition rule you gave me it is truly helpful. Thanks again! –  Cristian Jul 7 '12 at 6:14

The logic of computation looks okay to me. To alleviate your concerns about differentiating a functional, consider replacing $\rho$ with $\rho+h\psi$, where $\psi$ is another function (direction vector) and $h$ is a real variable. Holding both functions fixed, differentiate with respect to $h$ - this is the usual derivative of a real function. The result will be a linear functional of $\psi$, which is the derivative you are looking for. In particular, the derivative is zero if it is zero for every direction $\psi$ in your space of interest.

-
Thanks Leonid! If I got your point right, I should use $\rho +h\psi$ with $\psi$ an arbitrary function in the space of interest. Hence, I can pick $\psi=\rho$, can't I? One last thing, after differentiating, should I evaluate the result at $h=0$? Thank you! –  Cristian Jul 7 '12 at 4:40