Showing a function is Frechet Differentiable？ I just started learning the Frechet Derivatives.
So I have a function $H:\mathbb{R}^{N\times n}\to\mathbb{R}^{N\times n}$, i.e. $U^T\in\mathbb{R}^{N\times n}$ and 
$$H(U^T)=GW\times (F(U))^T+S\times U^T+C$$
with $G,W,S\in \mathbb{R}^{N \times N}$ are two matrices of size $N\times N$, $F(\cdot)\in \mathbb{R}^n\to\mathbb{R}^n $ is a nonlinear function which maps each column vetor of $U$ to the corresponding column vector of $F(U)$, and $C\in\mathbb{R}^{N\times n}$. 
My question is what property should the nonlinear unknown function $F(\cdot)$ satisfy to ensure the function $H(\cdot)$ is Frechet differentiable? What does the Frechet derivative matrix looks like? What should I start?Thank you!
 A: I'll rewrite the definition of $H$ as
$$
H(X) = GW  F(X^T)^T + SX + C.
$$
Let's assume that $F$ is Frechet differentiable at a particular point $X^T$,
so that
$$
F(X^T + \Delta X^T) = F(X^T) + F'(X^T) \Delta X^T + e(\Delta X),
$$ 
and the error term $e(\Delta X)$ satisfies
$$
\lim_{\Delta X \to 0} \frac{\|e(\Delta X)\|}{\| \Delta X \|} = 0.
$$
Notice that 
\begin{align}
H(X + \Delta X) &= GW F(X^T + \Delta X^T)^T + SX + S\Delta X + C \\
&= GW \left( F(X^T) + F'(X^T) \Delta X^T  + e(\Delta X) \right)^T + SX + S \Delta X + C \\
&= \underbrace{GWF(X^T)^T + SX + C}_{H(X)} + \underbrace{GW(F'(X^T) \Delta X^T)^T + S \Delta X}_{H'(X) \Delta X} + \underbrace{GW e(\Delta X)^T}_{\text{small}}.
\end{align}
Comparing this with the equation
$$
H(X + \Delta X) \approx H(X) + H'(X) \Delta X
$$
suggests that $H$ is differentiable at $X$ and that $H'(X)$ is the linear transformation defined by
$$
\tag{1} H'(X) \Delta X = GW(F'(X^T) \Delta X^T)^T + S \Delta X.
$$
To prove that this is true, we only need to show that
$$
\tag{2} \lim_{\Delta X \to 0} \frac{\| GW e(\Delta X)^T \|}{ \| \Delta X \|} = 0
$$
To establish (2), let $L$ be the linear transformation defined by
$$
L(v) = GW v^T.
$$ 
Then
\begin{align}
\frac{\| GW e(\Delta X)^T \|}{ \| \Delta X \|}
&= \frac{\| L(e(\Delta X)) \|}{\| \Delta X \|} \\
&\leq \frac{\| L \| \|e(\Delta X) \|}{\| \Delta X \|}
\end{align}
which approaches $0$ as $\Delta X \to 0$.
In order to reach the conclusion that $H$ is differentiable at $X$, we needed to assume that $F$ is differentiable at $X^T$.
I don't see a simpler way to express $H'(X)$, but maybe somebody else will.
A: They gave you the hand-wavy proof above. I am giving you the high-end proof now.
Recall from basic differential calculus:
Basic differentiation algebra: the derivative acts linearly $(f+ \alpha g)'(x) = f'(x) + \alpha g'(x)$; the derivative of constant functions is zero; and the derivative of continuous linear functions are themselves $f'(x) \cdot h = f(h)$ whenever $f$ is linear and continuous.
Chain rule: if $g$ and $f$ are two functions defined on open subsets of normed vector spaces such that $f$ is differentiable at $x$ and $g$ is differentiable at $f(x)$ then the composite function $g \circ f$ is differentiable at $c$ and its derivative is the composite of the derivatives $$(g \circ f)'(x) = g'(f(x)) \circ f'(x).$$
Abridged proof. Write $y = f(x)$ and $f(x + h) = f(x) + \underbrace{f'(x) \cdot h + o(h)}_k$ and $$g(y + k) = g(y) + g'(y) k + o(k) = g(y) + g'(y) \cdot f'(x) \cdot h + \underbrace{g'(y) o(h) + o(k)}_{o(k)}. \square$$

To your exercise. The function $H$ is differentiable at every $U$ where the function $F$ is differentiable as well.

Proof. The functions $\varphi:V \mapsto GW V^\intercal$ and $\psi = U \mapsto SU^\intercal$ are linear while the function $U \mapsto C$ is contant. Therefore, the function $H = \varphi \circ F + \psi + C$ will be differentiable at all points where $F$ is differentiable (by the chain rule) and its derivative is simply $$H'(U) = \varphi'(U) \circ F'(U)^\intercal + \psi'(U) = \varphi \circ F'(U) + \psi.$$
If you are dealing with finite dimensional vector spaces, find bases of each so that (by denoting $[ \cdot ]$ the matrix represantion) we get $$[H'(U)]=[\varphi][F'(U)]^\intercal + [\psi]. \square$$
Ammend. If the function $\varphi$ is invertible then the differentiability of $H$ implies that of $F$ for we can write $F = \varphi^{-1} \circ (H - \psi - C),$ and $\varphi^{-1}$ being linear and continuous, it is differentiable. $\square$
