Derivative of a functional w.r.t. a single point? Lets say I have a functional $J:F\to\mathbb R$ where $F$ is the space of continuous functions (or differentiable functions, if you want) on $\mathbb R$.
then let's say for a particular function $f:\mathbb R\to \mathbb R$, we calculate:
$$z=J(f)$$
Now suppose that I want to ask the following question: how much does $z$ change, if we perturb $f$ at a single point $a$?
So we kind of want:
$$\frac d {dy}J(f) \quad \text { where } f(a)=y$$
(This notation is probably not very good).
Is there a concept that captures this? I know that the Gateaux derivative looks at the change in $J$ that results from a perturbation w.r.t. to a function, but I'd like a concept that captures the change in $J$ as a result of a change in a specific point $a$. Moreover, we can then calculate this for the entire domain of $f$, and see how changes in the different values of $f$ result in changes in the total value of the functional. 
 A: There is a way you can make sense of this idea, but the notion of the derivative depends on a certain choice, which makes it non canonical.
The differential geometric view of a derivative is the following: Given a point $f\in F$ you define $$T_{f}F=\{\gamma{}\colon\mathbb{R}\rightarrow F \text{ differentiable}\vert ~\gamma_0=f\}/_\sim,$$
where you mod out the equivalence relation
$$ \gamma \sim\bar\gamma \quad \Leftrightarrow \quad \frac d{dt}\gamma{\big\vert_{t=0}} = \frac d{dt}\bar\gamma{\big\vert_{t=0}} .$$
$T_fF$ is the tangent space at $f$and should be viewed as space of all directions in which you can change $f$. Now the derivative of $J$ at $f$ will be a functional
$$
J_*\colon T_fF \rightarrow \mathbb{R}, \quad J_*([\gamma])=\frac d{dt}J\circ \gamma \big\vert_{t=0}
$$
You want to change $f$ at a single point $a$, so the question is whether this "direction if change" can be found in $T_fF$. You could for example put $\gamma^1_t(x)=f(x)+t g(x)$, where $g\in F$ is  a function with support only in a small neighbourhood of $a$. This defines an equivalence-class $[\gamma^1]\in T_fF$ and you might want to define
$$
\frac d{dy}J(f) := J_*[\gamma^1].
$$
But keep in mind, this will depend on the choice of $g$ that you make. For this to be a good definition for all possible $J$ one would hope that
$$
 \frac d{dt}\big\vert_{t=0}\gamma_t^1(a) = \frac d{dt}\big\vert_{t=0}\gamma_t^2(a) \quad \Rightarrow \quad \gamma^1 \sim \gamma^2 \quad \Rightarrow \quad J_*[\gamma^1] = J_*[\gamma^2],
$$
But it is clear that the first implication cannot be true. It is however a valid question, in which cases the implication
$$
 \frac d{dt}\big\vert_{t=0}\gamma_t^1(a) = \frac d{dt}\big\vert_{t=0}\gamma_t^2(a) \quad \Rightarrow   \quad J_*[\gamma^1] = J_*[\gamma^2],
$$
holds true. For example for $J_1(f)=f(a)$ or $J_2(f)=f'(a)$ it is true. And I think it should not be too hard to prove that any valid $J$ is an expression of these two.

EDIT 1: In an effort to find a canonical tangent vector one could try the following: Put $g^\sigma$ the normal distribution with mean $a$ and variance $\sigma^2$. Let $\gamma_t^\sigma=f+tg^\sigma$ be the associated curve, then you get a family of tangent vectors $([\gamma^\sigma])_\sigma\subset T_fF$. Does this have a limit in $T_fF$ as $\sigma \rightarrow 0$?
Unfortunately the answer is no. Usually one identifies $T_fF$ with $F$ via the linear isomorphism
$$
  T_fF\rightarrow F,\quad [\gamma] \mapsto \frac d{dt}\gamma
\big\vert_{t=0}$$
Now  $\frac d{dt}\gamma^\sigma
\big\vert_{t=0}=g^\sigma\in F$, but this does not have a limit in $F$ (equipped with the $\sup$-norm).

EDIT 2: The procedure described in the comments above invokes the Dirac Delta $\delta_a$ concentrated at $a$ and the question is whether you can extend $J$ to a larger function space $G$ containing $\delta_a$ in order to study the curve $t\mapsto f + t\delta_a$.
This question of extendability is a valid one, but cannot be answered in this generality, so it depends on which $J$ you are actually studying. For example the evaluation $Jf:=f(b)$ does not extend continously to distributions, not even to $L^2\mathbb{R}$. On the other hand the functional $Jf:=\int\phi f dx$ for a fixed $\phi\in C^\infty_c\mathbb{R}$ has an obvious extension to distributions. 
When dealing with differentiation it might also be favourable for $G$ to be a normed space (I don't know too much about how well behaved the above described procedure is in spaces with a more difficult topology). And indeed there are normed spaces that naturally contain $\delta_a$ (i.e. you don't need to extend to all distributions), namely Sobolov spaces $H^s\mathbb{R}$ with order $s$ below a certain (negative) threshold. Of course you are also interested in nonlinear $J$s, but in order to give you an idea which functionals can be extended to $H^s$, let me note that the continous dual of $H^s$ consists of those linear functionals which are integration against a weight $w\in H^{-s}$.
