Let $f$ be Lipschitz around $a \in \mathbb R^d$. Then $f$ is Fréchet differentiable at $a$ if and only if $f$ is Gâteaux differentiable at $a$ This thread is meant to record a question that I feel interesting during my self-study. I'm very happy to receive your suggestion and comments.
See: SE blog: Answer own Question and MSE meta: Answer own Question. Anyway, it is written as problem.

Let $X$ be a normed space, $A \subset X$ an open set, $f: A \rightarrow \mathbb{R}$ a function, and $a \in A$ a point. For a "direction" $v \in X$, we shall consider the directional derivative $f^{\prime}(a, v)$, which are defined by:
$$
\begin{aligned}
f^{\prime}(a, v) &=\lim _{t \rightarrow 0} \frac{f(a+t v)-f(a)}{t}
\end{aligned}
$$
We shall say that $f$ is:

*

*Gâteaux differentiable at $a$ if $f^{\prime}(a, \cdot) \in X^{*}$, i.e., $f^{\prime}(a, \cdot)$ is everywhere defined, real-valued, linear and continuous);

*Fréchet differentiable at $a$ if there exists $x^{*} \in X^{*}$ such that
$$
\lim _{\|h\| \rightarrow 0} \frac{f(a+h)-f(a)-x^{*}(h)}{\|h\|}=0 .
$$

Theorem: Let $X, A, f, a$ be as above. Assume that $X=\mathbb{R}^{d}$ and $f$ is Lipschitz on some neighborhood of $a$. Then $f$ is Fréchet differentiable at $a$ if and only if $f$ is Gâteaux differentiable at $a$.

 A: Assume the contrary that $f$ is not Fréchet differentiable at $a$, i.e., there exists a sequence $(h_n) \subset X$ such that $h_n \neq 0$ and $h_n \to 0$, and that
$$
D_n := \frac{f(a+h_n) - f(a)- f'(a, h_n)}{\|h_n\|} \not \to 0.
$$
This implies there is $\varepsilon>0$ and a subsequence $\varphi$ such that
$$
|D_{\varphi(n)}| \ge \varepsilon \quad \forall n.
$$
Let $t_n := \|h_n\|$ and $v_n := h_n/t_n$. Then $\|v_n\|=1$ and $t_n \to 0$. The unit sphere of $X$ is compact, so we can assume there is $v\in X$ and a subsequence $\psi$ of $\varphi$ such that $v_{\psi(n)} \to v$. We have
$$
\begin{align}
D_{\psi(n)} &= \frac{f(a+t_{\psi(n)} v_{\psi(n)}) - f(a)- t_{\psi(n)} f'(a, v_{\psi(n)})}{t_{\psi(n)}} \\
&= \frac{f(a+t_{\psi(n)} v) - f(a)- t_{\psi(n)} f'(a, v)}{t_{\psi(n)}}  +\frac{f(a+t_{\psi(n)} v_{\psi(n)}-f(a+t_{\psi(n)} v)}{t_{\psi(n)}} + f'(a, v) - f'(a, v_{\psi(n)}).
\end{align}
$$
WLOG, we can assume $f$ is $L$-Lipschitz on $A$. Then
$$
|D_{\psi(n)} | \le \left |\frac{f(a+t_{\psi(n)} v) - f(a)- t_{\psi(n)} f'(a, v)}{t_{\psi(n)}} \right| + L\|v_{\psi(n)}-v\| + \|f'(a, \cdot)\| \|v_{\psi(n)}-v\|.
$$
It follows that $D_{\psi(n)} \to 0$. This is a contradiction and thus completes the proof.
