# Showing a function is Frechet Differentiable？

I just started learning the Frechet Derivatives. So I have a function $$H:\mathbb{R}^{N\times n}\to\mathbb{R}^{N\times n}$$, i.e. $$U^T\in\mathbb{R}^{N\times n}$$ and $$H(U^T)=GW\times (F(U))^T+S\times U^T+C$$ with $$G,W,S\in \mathbb{R}^{N \times N}$$ are two matrices of size $$N\times N$$, $$F(\cdot)\in \mathbb{R}^n\to\mathbb{R}^n$$ is a nonlinear function which maps each column vetor of $$U$$ to the corresponding column vector of $$F(U)$$, and $$C\in\mathbb{R}^{N\times n}$$.

My question is what property should the nonlinear unknown function $$F(\cdot)$$ satisfy to ensure the function $$H(\cdot)$$ is Frechet differentiable? What does the Frechet derivative matrix looks like? What should I start?Thank you!

• What does $\times$ mean here? Dec 2, 2018 at 21:28
• @WillM. Sorry for the confusing notation. It means normal matrix multiplication. Dec 3, 2018 at 16:11
• "with $G,W,S\in \mathbb{R}^{N \times N}$ are two matrices" did you meant three instead of two? are these constant matrices?
– zhw.
Dec 5, 2018 at 19:52

I'll rewrite the definition of $$H$$ as $$H(X) = GW F(X^T)^T + SX + C.$$ Let's assume that $$F$$ is Frechet differentiable at a particular point $$X^T$$, so that $$F(X^T + \Delta X^T) = F(X^T) + F'(X^T) \Delta X^T + e(\Delta X),$$ and the error term $$e(\Delta X)$$ satisfies $$\lim_{\Delta X \to 0} \frac{\|e(\Delta X)\|}{\| \Delta X \|} = 0.$$ Notice that \begin{align} H(X + \Delta X) &= GW F(X^T + \Delta X^T)^T + SX + S\Delta X + C \\ &= GW \left( F(X^T) + F'(X^T) \Delta X^T + e(\Delta X) \right)^T + SX + S \Delta X + C \\ &= \underbrace{GWF(X^T)^T + SX + C}_{H(X)} + \underbrace{GW(F'(X^T) \Delta X^T)^T + S \Delta X}_{H'(X) \Delta X} + \underbrace{GW e(\Delta X)^T}_{\text{small}}. \end{align}

Comparing this with the equation $$H(X + \Delta X) \approx H(X) + H'(X) \Delta X$$ suggests that $$H$$ is differentiable at $$X$$ and that $$H'(X)$$ is the linear transformation defined by $$\tag{1} H'(X) \Delta X = GW(F'(X^T) \Delta X^T)^T + S \Delta X.$$ To prove that this is true, we only need to show that $$\tag{2} \lim_{\Delta X \to 0} \frac{\| GW e(\Delta X)^T \|}{ \| \Delta X \|} = 0$$ To establish (2), let $$L$$ be the linear transformation defined by $$L(v) = GW v^T.$$ Then \begin{align} \frac{\| GW e(\Delta X)^T \|}{ \| \Delta X \|} &= \frac{\| L(e(\Delta X)) \|}{\| \Delta X \|} \\ &\leq \frac{\| L \| \|e(\Delta X) \|}{\| \Delta X \|} \end{align} which approaches $$0$$ as $$\Delta X \to 0$$.

In order to reach the conclusion that $$H$$ is differentiable at $$X$$, we needed to assume that $$F$$ is differentiable at $$X^T$$.

I don't see a simpler way to express $$H'(X)$$, but maybe somebody else will.

• Thanks for your help! I have a question, what is the form of $F'(X^T)^T$? Is it a matrix? Is it possible to represent it explicitly using derivatives of $f_i, 1\le i \le n$ suppose $F=(f_1,\ldots,f_n)$? Dec 3, 2018 at 17:03

They gave you the hand-wavy proof above. I am giving you the high-end proof now.

Recall from basic differential calculus:

Basic differentiation algebra: the derivative acts linearly $$(f+ \alpha g)'(x) = f'(x) + \alpha g'(x)$$; the derivative of constant functions is zero; and the derivative of continuous linear functions are themselves $$f'(x) \cdot h = f(h)$$ whenever $$f$$ is linear and continuous.

Chain rule: if $$g$$ and $$f$$ are two functions defined on open subsets of normed vector spaces such that $$f$$ is differentiable at $$x$$ and $$g$$ is differentiable at $$f(x)$$ then the composite function $$g \circ f$$ is differentiable at $$c$$ and its derivative is the composite of the derivatives $$(g \circ f)'(x) = g'(f(x)) \circ f'(x).$$ Abridged proof. Write $$y = f(x)$$ and $$f(x + h) = f(x) + \underbrace{f'(x) \cdot h + o(h)}_k$$ and $$g(y + k) = g(y) + g'(y) k + o(k) = g(y) + g'(y) \cdot f'(x) \cdot h + \underbrace{g'(y) o(h) + o(k)}_{o(k)}. \square$$

To your exercise. The function $$H$$ is differentiable at every $$U$$ where the function $$F$$ is differentiable as well.

Proof. The functions $$\varphi:V \mapsto GW V^\intercal$$ and $$\psi = U \mapsto SU^\intercal$$ are linear while the function $$U \mapsto C$$ is contant. Therefore, the function $$H = \varphi \circ F + \psi + C$$ will be differentiable at all points where $$F$$ is differentiable (by the chain rule) and its derivative is simply $$H'(U) = \varphi'(U) \circ F'(U)^\intercal + \psi'(U) = \varphi \circ F'(U) + \psi.$$

If you are dealing with finite dimensional vector spaces, find bases of each so that (by denoting $$[ \cdot ]$$ the matrix represantion) we get $$[H'(U)]=[\varphi][F'(U)]^\intercal + [\psi]. \square$$

Ammend. If the function $$\varphi$$ is invertible then the differentiability of $$H$$ implies that of $$F$$ for we can write $$F = \varphi^{-1} \circ (H - \psi - C),$$ and $$\varphi^{-1}$$ being linear and continuous, it is differentiable. $$\square$$