# Gâteaux derivate of the Tikhonov functional

Let $X,Y$ be Hilbert spaces, and let $A\colon X\to Y$ be a compact operator. The Tikhonov functional is given by $$F(x)=\lVert Ax-y\rVert_X^2+\alpha\lVert x\rVert_X^2.$$ Calculate the Gâteaux derivative of the Tikhonov functional.

Tip: Use $\lVert a+b\rVert=\lVert a\rVert^2+2(a,b)+\lVert b\rVert^2$.

First question: Is it really $\lVert Ax-y\rVert_X^2$ and not the Y-norm?

Second question: What I have to do here is to my opinion calculate

$$\lim\limits_{t\to 0}\frac{F(x+th)-F(x)}{t}.$$

So I would start with calculating $F(x+th)=\lVert A(x+th)-y)\rVert_X^2+\alpha\lVert x+th\rVert_X^2$.

I start with calculating $\lVert A(x+th)-y)_X^2$. I can not read from the text if the operator $A$ is linear, too. Is it right to calculate with the given tip

$$\lVert A(x+th)-y\rVert_X^2=\lVert A(th)+Ax-y\rVert_X^2=\lVert A(th)\rVert_X^2+2(A(th),Ax-y)+\lVert Ax-y\rVert_X^2?$$

-
The first norm should be in $Y$, not in $X$. In your definition of the Tikhonov functional you may also want to specify what $y$ is. I assume it is a constant vector in $Y$. – Willie Wong Feb 1 '13 at 13:54
And yes, the notations and terminology suggest that $A$ is linear. – Willie Wong Feb 1 '13 at 13:55
My result then is: The Gâteaux derivate in $x$ in direction $h$ is given by $DF(x)[h]=2\cdot(h,(A^*A+\alpha\mbox{id})x-Ay)_X$. – math12 Feb 1 '13 at 14:34

Since $Ax$ and $y$ are both in $Y$, the first norm should indeed be the norm in $Y$, not in $X$. Compact operator is (often, at least) implicitly assumed to be linear and the notation $Ax$ instead of $A(x)$ also suggests linearity.
Let us calculate the derivative of $F$ at $x\in X$. I will assume that your Hilbert spaces are over the reals; the complex version is similar. For any $h\in X$ we have $$\begin{split} F(x+h) &= \|A(x+h)-y\|_Y^2+\alpha\|x+h\|_X^2 \\&= \|Ax-y\|_Y^2+\|Ah\|_Y^2-2\langle Ax-y,Ah\rangle_Y+\alpha\|x\|_X^2+\alpha\|h\|_X^2+2\alpha\langle x,h\rangle_X \\&= F(x)+2\alpha\langle x,h\rangle_X-2\langle Ax-y,Ah\rangle_Y+\|Ah\|_Y^2+\alpha\|h\|_X^2 \\&= F(x)+2\alpha\langle x,h\rangle_X-2\langle A^*(Ax-y),h\rangle_X+\|Ah\|_Y^2+\alpha\|h\|_X^2 \\&= F(x)+2\langle\alpha x-A^*Ax+A^*y,h\rangle_X+\|Ah\|_Y^2+\alpha\|h\|_X^2. \end{split}$$ The Gâteaux derivative of $F$ at $x$ in direction $h$ is $$\begin{split} D_GF(x)h &= \lim_{t\to0}\frac{F(x+ht)-F(x)}{t} \\&= \lim_{t\to0}\left(2\langle\alpha x-A^*Ax+A^*y,h\rangle_X+t\|Ah\|_Y^2+t\alpha\|h\|_X^2\right) \\&= 2\langle\alpha x-A^*Ax+A^*y,h\rangle_X. \end{split}$$ The Gâteaux derivative $D_GF(x)$ is an element of $X^*$ which can be identified with $X$ via the inner product. After this identification $$D_GF(x) = 2(\alpha x-A^*Ax+A^*y).$$ In fact, since $A$ is continuous (follows from compactness), the function $F$ is actually Fréchet differentiable and the two derivatives agree.
If $A$ were not linear, the derivative would include a derivative of $A$. I assume nonlinearity was not intended in your exercise.