Derivative of the integral with respect to the function Consider this function:
$$ E[L] = \int\int\{ y(x) - t \}^2p(x,t)dx dt $$
I try to figure out how to take the derivate of this function with respect to $y(x)$.
In the book it is:
$$ \frac{ \delta E[L] }{ \delta y(x)} = 2 \int \{ y(x) - t\}p(x,t)dt $$
This I cannot understand. My initial thought was to use chain-rule of the derivative and push derivative operator under integral, but I cannot achieve the same result.
Thanks in advance!
 A: The functional derivative of $E$ at $y(x)$ is defined by the equation
$$ \int \frac{\delta F}{\delta y}(x) \phi(x) \, dx = \frac{d}{ds} E(y(x) + s\phi(x))|_{s = 0}. $$
That is, for each variation $\phi$, you consider the directional derivative $\frac{d}{ds} E(y(x) + s\phi(x))|_{s = 0}$ (which is a number) and you need to find a function $\frac{\delta F}{\delta y}(x)$ such that integration of it against the variation $\phi$ gives you the directional derivative.
In your example,
\begin{align*}
\frac{d}{ds} E(y(x) + s\phi(x))|_{s = 0} 
&= \frac{d}{ds} \left( \iint \{ y(x) + s\phi(x) - t\}^2 p(x,y) \, dx \, dt \right)|_{s=0} \\
&= \frac{d}{ds} \left( \iint \{ (y(x) - t)^2 + 2s(y(x)-t)\phi(x) + s^2\phi^2(x) \} p(x,t) \, dx \, dt \right)|_{s=0} \\
&= \frac{d}{ds} \bigg( \iint \{y(x) - t\}^2 p(x,t) \, dx \, dt \\
&\qquad+s \iint \{ 2(y(x) - t)\phi(x) \} p(x,t) \, dx \, dt \\
&\qquad +s^2 \iint \{ \phi^2(x) \} p(x,t) \, dx \, dt  \bigg)|_{s=0} \\
&= 2 \iint \{(y(x) - t)\phi(x) \} p(x,t) \, dx \, dt\\
& = \int \left( 2 \int \{ y(x) - t \} p(x,t) \, dt \right) \phi(x) \, dx
\end{align*}
showing that
$$ \frac{\delta F}{\delta y}(x) = 2 \int \{ y(x) - t \} p(x,t) \, dt. $$
