# Proving Fréchet differentiability

Am learning about Fréchet differentials and was wondering if for a real matrix $X$ and positive semidefinite real matrices $A,B$ the function $f(X)=TrX^TAX-X^TBX$ is twice Fréchet differentiable or not?

I believe that the gradient is $2AX-2BX$ but how do I prove Fréchet differentiability?

-
If all 1st order partial derivatives are continuous, the function is Fréchet differentiable. – user31373 Sep 14 '12 at 2:50

$\def\Mat{\mathop{\mathrm{Mat}}\nolimits}\def\R{\mathbb R}\def\tr{\mathop{\mathrm{tr}}}\def\abs#1{\left|#1\right|}\def\norm#1{\left\|#1\right\|}$ In order for $f\colon \Mat_n(\R) \to \R$ to be differentiable at $X \in \Mat_n(\R)$, it must hold $f(X + H) = f(X) + f'(X)H + o(H), \quad H \to 0$ for some linear map $f'(X)\colon \Mat_n(\R) \to \R$. Just computing, we get (writing $C := A -B$ for brevity and using $C^T = C$) \begin{align*} f(X + H) &= \tr(X^TCX + H^TCX + X^TCH + H^TCH)\\ &= \tr(X^TCX) + 2\tr(H^TCX) + \tr(H^TCH) \end{align*} We have \begin{align*} \abs{\tr H^TCH} &\le n\norm{H^TCH}_2\\ &\le n\norm{C}_2\norm{H}^2_2 \end{align*} so $\tr(H^TCH) = o(H), H \to 0$ and $f'(X)H = 2\tr(H^TCX)$.

For showing that $f$ is twice differentiable, we have to prove that the map $f'\colon \Mat_n(\R) \to \Mat_n(\R)^*, \quad X \mapsto 2\tr({-}^TCX)$ is differentiable at each $X$, that is, there is a linear $f''(X)\colon \Mat_n(\R) \to \Mat_n(\R)^*$ with $f'(X+K) = f'(X) + f''(X)K + o(K), \quad K \to 0$ But this is easy, as $f'$ is linear, so if we define $f''(X) = f'$ for each $X$, we have $f'(X+K) = f'(X) + f'(K) = f'(X) + f''(X)K.$ So, $f$ is twice (Frechet) differentiable.

Another way to show your claim is to follow LVKs comment and show that all partial derivatives of $f$ up to order 2 exist and are continuous (which implies that $f$ is twice differentiable by a well known theorem), we have, writing $[M]_{ij}$ for the $i,j$-th coordinate of a matrix $M$ and $X = (x_{ij})$: \begin{align*} f(X) &= \sum_i [X^TCX]_{ii}\\ &= \sum_{i,j,k} [X^T]_{ij}[C]_{jk}[X]_{ki}\\ &= \sum_{i,j,k} [C]_{jk}x_{ji}x_{ki} \end{align*} As this is a polynomial in the $x_{ij}$, its partial derivatives are polynomials themselves, so your claim follows again.

-
What does $2\tr({-}^TCX)$ mean in $quad X \mapsto 2\tr({-}^TCX)$? i.e, What is $-^T$? – user23600 Sep 14 '12 at 14:39
As $f'(X) = 2\mathrm{tr}(-^TCX)$ is an element of $\mathrm{Mat}_n(\mathbb R)^*$, it is an linear mapping. The $-$ is a placeholder for the argument, it is $2\mathrm{tr}(-^TXC)H = 2\mathrm{tr}(H^TCX)$. – martini Sep 14 '12 at 18:24
Ah! Got it-Similar to a dot or an asterisk. – user23600 Sep 14 '12 at 19:02
Is $\mathbb{R}^*$ a ring or a dual space in this context? Does it denote the space of invertible real matrices? What space is this? – user23600 Sep 14 '12 at 20:58
$\mathrm{Mat}_n(\mathbb R)^*$ denotes the dual space of $\mathrm{Mat}_n(\mathbb R)$, i. e. the linear functionals on the matrices. – martini Sep 17 '12 at 9:20