# What is the Fréchet derivative?

I'm sorry if I sound too ignorant, i don't have a high level of knowledge in math

I've been lately trying to understand the Frèchet derivative. I'm just starting Calc II, but I have a tiny grasp in multivariable calculus. That is, I understand that one can treat some of the variables as constants and get a directional derivative.

According to wikipedia, if the limit of the equation below as $h$ tends to $0$ is equal to $0$ then the function is said to be Fréchet differentiable at $x$.

$$\frac{||f(x+h)-f(x)-T(x)||}{||h||}$$

And as much as this can work as a definition, I'm still wondering what the Fréchet derivative actually is.

My confusion might also come from the fact that there are some ideas in multi-variable calculus I don't get, so I will try to summarize my questions below.

Q1. If I understood correctly, $f$ could be a multi-variable function, so how come we use only one $x$? Is it because $x$ is itself a multi-variable vector?

Q2. What is the meaning of $T(h)$? As in, what is it, and what is it doing in this equation?

Q3. Why does this equation -when $h$ tends to $0$- somehow tells whether a function is (Fréchet) differentiable or not? How does this equation relates to what a Fréchet derivative is?

Any help/thoughts would be really appreciated.

• $T(h)$ is a linear function in $h$. If $f$ is scalar valued, $T$ is the gradient of $f$ at $x$, and $T(h)$ is the directional derivative of $f$ at $x$ in the direction $h$, $\nabla f \cdot h$, if you will. If $f$ is vector-valued, $T$ is a matrix of partial derivatives and $T(h)$ is that matrix times the vector $h$. – kimchi lover Aug 21 '17 at 2:09
• $T$ is a linear mapping. Frechet is stronger than just possessing directional derivative, see example math.stackexchange.com/questions/2390373/… – Will Jagy Aug 21 '17 at 2:21
• Perhaps lectures 21-24 of my course on YouTube will help. Ultimately, you may wish to watch others. – Ted Shifrin Aug 21 '17 at 5:22

Q1: Yeah, $x$ is a vector; in general, we speak of $f : \mathbb{R}^n \to \mathbb{R^m}$, in which case $x \in \mathbb{R}^n$ and $f(x) \in \mathbb{R}^m$.

Q2: This was one of the coolest and hardest-to-grasp things (for me personally) when I first encountered multivariable. I'll try to sum it up here. In short: the derivative is not a number. Not anymore; in multivariable, the derivative is a linear transformation (and in single-variable, too, since this is a generalization; but in single-variable, the idea that the derivative is a number works fine - in multivariable, it doesn't make much sense). You can think of it as the best linear approximation to $f$; $T$, the derivative of $f$ at a point, is a linear function now.

Q3: The condition that this limit is zero means that this linear function $T$ is in fact a "first order" approximation. That is, $$f(x + h) = f(x) + T(h) + \text{error term},$$ where the error term tends to zero faster than $|h|$ does. This is just like the Taylor expansion in single variables.

Hope this helps. As a relation to single-variable, you can think of the usual derivative of some $f: \mathbb{R} \to \mathbb{R}$ as a linear transformation as follows.

The usual derivative of $f$ at some point $a$ is $f'(a)$, a number. In calc, you learn that the tangent line $y = f'(a)(x - a) + f(a)$ is the best linear approximation to the function $f$ near $a$. We can transform this into this new concept as follows: define $T : \mathbb{R} \to \mathbb{R}$ by $$T(h) = f'(a)\cdot h.$$ Then $T$ is a linear transformation that takes a "vector" (in this case a single number) and returns $f'(a)$ times that vector. Note also that $$\lim_{h \to 0}\frac{f(a + h) - f(a) - T(h)}{h} = \lim_{h \to 0}\frac{f(a + h) - f(a) - f'(a)h}{h}$$ $$=\lim_{h \to 0}\frac{f(a + h) - f(a)}{h} - f'(a) = 0$$ this last line following from splitting up the fraction, distributing the limit, and recognizing (by the "normal" definition of the derivative) that this limit is by definition f'(a).

The previous answer is fine. $T(h)$ is the linear term in the Taylor expansion of $f$ about $x$ in the direction $h$.

Perhaps this discussion might add to the previous answer with regards to Q3. The ratio that tends to 0 measures the error of the Taylor expansion $f(x+h) = f(x) + T(h) + R(x,h)$. Any theory of differentiation will have $R(x,h)\to 0$ as $h\to 0$, faster than $h\to0$, ($R = o(h)$, if you will) and the different kinds of differentiation (Gâteaux, Hadamard, Fréchet) differ only in the way in which $R$ is required to tend to 0. In particular, they ask for $R(x,th)/t\to0$ as $t\in\mathbb R$ tends to 0, pointwise for each $h$, uniformly in $h$ in compact sets, and uniformly in $h$ in bounded sets, respectively. (In finite dimensional spaces these last two coincide, but not in general.) By analogy with the 1 dimensional case, one wants $R(x,h)=o(h)$, to be of smaller order than $h$. But one cannot divide by vectors, and the different kinds of derivatives are different work-arounds. The Fréchet one corresponds to making $R=o(\|h\|)$.

For day to day practical work, Fréchet differentiability is the useful concept.