"Taylor Series" analog for functionals? For a function $f(x)$, it is possible to write it as a taylor series centered around a point $x=a$:
$$f(x)=\sum_{n=0}^{\infty}\frac{f^{(n)}(a){(x-a)}^{n}}{n!}=f(a)+f'(a)(x-a)+\frac{f''(a)(x-a)^{2}}{2}+...$$
(Of course, there's a lot more mathematical nuance to Taylor Series expansion, I just want to lay it out loosely here as a basis for my intuition.)
I'm wondering if there's anyway to apply this to functionals, that is, a functional $F[f(x)]$ that maps the function $f(x)$ to an output. Is there a way to "rewrite" a functional as a series such as this:
$$F[f(x)]=a_0+a_1(f(x)-\phi(x))+a_2(f(x)-\phi(x))^2+...$$
Where $\phi(x)$ is a function that acts analogously to the point $x=a$  in a Taylor expansion. 
(Again, I'm using all of my terminology and notation pretty loosely here. I'm not going for robust mathematical rigoorousness; I just want to express my intuition behind this idea.)
Is this "Functional expanded as a series" idea a thing? What is it called? Does it have any applications?
 A: Indeed there is. This is used in calculus of variation. Commonly up to and including order two. See https://en.wikipedia.org/wiki/Functional_derivative
A: It seems counter intuitive, but if you forget functionals (That is functions from functions to numbers) and just look at functions from functions to functions) you can build a taylor series like idea somewhat intuitively. I'm still working out the details of it, but i have been able to use it to show for example
$$ f(x+1) = f + f' + \frac{1}{2!}f''+\frac{1}{3!}f''' + ... $$
See here: How to build taylor series for infinite dimensional objects? for how to construct such series and for a list of problems that I encounter.
Somehow, most ideas in calculus generalize very cleanly in the context of "functions of functions" but fail to generalize EASILY (not saying they dont at all) in the context of functionals.
A: The expansion of a functional around a function is drastically more complicated than suggested in the question and in other answers here so far.
Wheras the Taylor series of $f(x)$ around point $x_0$ is written simply as:
\begin{alignat}{2}
f(x) &= \sum_{n=0}^\infty c_n(x-x_0)^n &~~~,~~~c_n = \frac{f^{(n)}(x_0)}{n!}\tag{1}~,
\end{alignat}
a Volterra series expansion of $f[x(r)]$ around the function $x_0(r)$ is significantly more complicated, as it requires integration over a dummy-variable version of $r$, an additional time for each successive term in the series):
\begin{align}
f[x(r)] &= \sum_{n=0}^\infty \int \int \cdots \int c_n\left(\prod_{i=0}^n \left(x(r)-x_0(r^{(i)})\right) \right) \textrm{d}r^\prime   \textrm{d}r^{\prime \prime} \cdots \textrm{d}r^{(n)}\tag{2}\\
&=\int \sum_{n=0}^\infty c_n \prod_{i=0}^n \left(x(r)-x_0(r^{(i)})\right)     \textrm{d}r^{(i)}\tag{3}\\
 c_n &=\frac{f^{(n)}[x_0(r)]}{n!}\tag{4}\\
&= \frac{1}{n!}\frac{\delta f[x_0(r)]}{\delta f[x(r^\prime)]\delta f[x(r^{\prime\prime})]\cdots \delta f[x(r^{(n)})]}. \tag{5}\\
\end{align}
Eq. 2 has been used in practical applications, for example in this 1994 paper in which $r$ represented a 3-dimensional vector so each integral in Eq. 2 was in fact a triple integral. Chapter 5 of this classical field theory book (PDF) by Nicholas Wheeler also has a lot of information and analogies about the above expansions.
I'll write the expansion out for you one more time in a way that might make it even more clear to you how this expansion compares with the Taylor expansion with which you are probably very familiar:
$$\tag{6}{\small
f[x(r)] = f[x_0(r)] + \int \frac{f^{(1)}[x_0(r)]}{1!}\left(x(r)-x_0(r^\prime\right))\textrm{d}r^\prime +  \int\!\!\!\int \frac{f^{(2)}[x_0(r)]}{2!}\left(x(r)-x_0(r^\prime\right))^{(2)}\textrm{d}r^\prime\textrm{d}r{^{\prime\prime}} + \cdots,} 
$$
in which:
$$
\left(x(r)-x_0(r^\prime\right))^{(2)}\equiv\left(x(r)-x_0(r^\prime\right))\left(x(r)-x_0(r^{\prime\prime}\right)).
$$
This expansion has been the subject of my first MathOverflow question (earlier today!) which was about how to extend the Volterra series to functionals of more arbitrary complex-valued functions much like the Laurent series extends the concept of a Taylor series to more arbitrary functions of complex variables:

*

*“Taylor series” is to “Volterra series” as “Laurent series” is to _________?
I also answered today my first MathOverflow question, which was a 2.5 year old unanswered question about how to extend the Pade approximant in this way:

*

*“Taylor series” is to “Volterra series” as “Pade approximant” is to _________?
Some other Mathematics.SE questions which may interest you are:

*

*Functional Taylor series

*Taylor series with functions as parameters (as opposed to variables)

*Taylor expansion of functional
