Let $X,Y$ be Hilbert spaces, and let $A\colon X\to Y$ be a compact operator. The Tikhonov functional is given by $$ F(x)=\lVert Ax-y\rVert_X^2+\alpha\lVert x\rVert_X^2. $$ Calculate the Gâteaux derivative of the Tikhonov functional.

Tip: Use $\lVert a+b\rVert=\lVert a\rVert^2+2(a,b)+\lVert b\rVert^2$.

First question: Is it really $\lVert Ax-y\rVert_X^2$ and not the Y-norm?

Second question: What I have to do here is to my opinion calculate

$$ \lim\limits_{t\to 0}\frac{F(x+th)-F(x)}{t}. $$

So I would start with calculating $F(x+th)=\lVert A(x+th)-y)\rVert_X^2+\alpha\lVert x+th\rVert_X^2$.

I start with calculating $\lVert A(x+th)-y)_X^2$. I can not read from the text if the operator $A$ is linear, too. Is it right to calculate with the given tip

$$\lVert A(x+th)-y\rVert_X^2=\lVert A(th)+Ax-y\rVert_X^2=\lVert A(th)\rVert_X^2+2(A(th),Ax-y)+\lVert Ax-y\rVert_X^2?$$

  • $\begingroup$ The first norm should be in $Y$, not in $X$. In your definition of the Tikhonov functional you may also want to specify what $y$ is. I assume it is a constant vector in $Y$. $\endgroup$ – Willie Wong Feb 1 '13 at 13:54
  • $\begingroup$ And yes, the notations and terminology suggest that $A$ is linear. $\endgroup$ – Willie Wong Feb 1 '13 at 13:55
  • $\begingroup$ My result then is: The Gâteaux derivate in $x$ in direction $h$ is given by $DF(x)[h]=2\cdot(h,(A^*A+\alpha\mbox{id})x-Ay)_X$. $\endgroup$ – math12 Feb 1 '13 at 14:34

Since $Ax$ and $y$ are both in $Y$, the first norm should indeed be the norm in $Y$, not in $X$. Compact operator is (often, at least) implicitly assumed to be linear and the notation $Ax$ instead of $A(x)$ also suggests linearity.

Let us calculate the derivative of $F$ at $x\in X$. I will assume that your Hilbert spaces are over the reals; the complex version is similar. For any $h\in X$ we have $$ \begin{split} F(x+h) &= \|A(x+h)-y\|_Y^2+\alpha\|x+h\|_X^2 \\&= \|Ax-y\|_Y^2+\|Ah\|_Y^2-2\langle Ax-y,Ah\rangle_Y+\alpha\|x\|_X^2+\alpha\|h\|_X^2+2\alpha\langle x,h\rangle_X \\&= F(x)+2\alpha\langle x,h\rangle_X-2\langle Ax-y,Ah\rangle_Y+\|Ah\|_Y^2+\alpha\|h\|_X^2 \\&= F(x)+2\alpha\langle x,h\rangle_X-2\langle A^*(Ax-y),h\rangle_X+\|Ah\|_Y^2+\alpha\|h\|_X^2 \\&= F(x)+2\langle\alpha x-A^*Ax+A^*y,h\rangle_X+\|Ah\|_Y^2+\alpha\|h\|_X^2. \end{split} $$ The Gâteaux derivative of $F$ at $x$ in direction $h$ is $$ \begin{split} D_GF(x)h &= \lim_{t\to0}\frac{F(x+ht)-F(x)}{t} \\&= \lim_{t\to0}\left(2\langle\alpha x-A^*Ax+A^*y,h\rangle_X+t\|Ah\|_Y^2+t\alpha\|h\|_X^2\right) \\&= 2\langle\alpha x-A^*Ax+A^*y,h\rangle_X. \end{split} $$ The Gâteaux derivative $D_GF(x)$ is an element of $X^*$ which can be identified with $X$ via the inner product. After this identification $$ D_GF(x) = 2(\alpha x-A^*Ax+A^*y). $$ In fact, since $A$ is continuous (follows from compactness), the function $F$ is actually Fréchet differentiable and the two derivatives agree.

If $A$ were not linear, the derivative would include a derivative of $A$. I assume nonlinearity was not intended in your exercise.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.