Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $X,Y$ be Hilbert spaces, and let $A\colon X\to Y$ be a compact operator. The Tikhonov functional is given by $$ F(x)=\lVert Ax-y\rVert_X^2+\alpha\lVert x\rVert_X^2. $$ Calculate the Gâteaux derivative of the Tikhonov functional.

Tip: Use $\lVert a+b\rVert=\lVert a\rVert^2+2(a,b)+\lVert b\rVert^2$.

First question: Is it really $\lVert Ax-y\rVert_X^2$ and not the Y-norm?

Second question: What I have to do here is to my opinion calculate

$$ \lim\limits_{t\to 0}\frac{F(x+th)-F(x)}{t}. $$

So I would start with calculating $F(x+th)=\lVert A(x+th)-y)\rVert_X^2+\alpha\lVert x+th\rVert_X^2$.

I start with calculating $\lVert A(x+th)-y)_X^2$. I can not read from the text if the operator $A$ is linear, too. Is it right to calculate with the given tip

$$\lVert A(x+th)-y\rVert_X^2=\lVert A(th)+Ax-y\rVert_X^2=\lVert A(th)\rVert_X^2+2(A(th),Ax-y)+\lVert Ax-y\rVert_X^2?$$

share|cite|improve this question
The first norm should be in $Y$, not in $X$. In your definition of the Tikhonov functional you may also want to specify what $y$ is. I assume it is a constant vector in $Y$. – Willie Wong Feb 1 '13 at 13:54
And yes, the notations and terminology suggest that $A$ is linear. – Willie Wong Feb 1 '13 at 13:55
My result then is: The Gâteaux derivate in $x$ in direction $h$ is given by $DF(x)[h]=2\cdot(h,(A^*A+\alpha\mbox{id})x-Ay)_X$. – math12 Feb 1 '13 at 14:34

Since $Ax$ and $y$ are both in $Y$, the first norm should indeed be the norm in $Y$, not in $X$. Compact operator is (often, at least) implicitly assumed to be linear and the notation $Ax$ instead of $A(x)$ also suggests linearity.

Let us calculate the derivative of $F$ at $x\in X$. I will assume that your Hilbert spaces are over the reals; the complex version is similar. For any $h\in X$ we have $$ \begin{split} F(x+h) &= \|A(x+h)-y\|_Y^2+\alpha\|x+h\|_X^2 \\&= \|Ax-y\|_Y^2+\|Ah\|_Y^2-2\langle Ax-y,Ah\rangle_Y+\alpha\|x\|_X^2+\alpha\|h\|_X^2+2\alpha\langle x,h\rangle_X \\&= F(x)+2\alpha\langle x,h\rangle_X-2\langle Ax-y,Ah\rangle_Y+\|Ah\|_Y^2+\alpha\|h\|_X^2 \\&= F(x)+2\alpha\langle x,h\rangle_X-2\langle A^*(Ax-y),h\rangle_X+\|Ah\|_Y^2+\alpha\|h\|_X^2 \\&= F(x)+2\langle\alpha x-A^*Ax+A^*y,h\rangle_X+\|Ah\|_Y^2+\alpha\|h\|_X^2. \end{split} $$ The Gâteaux derivative of $F$ at $x$ in direction $h$ is $$ \begin{split} D_GF(x)h &= \lim_{t\to0}\frac{F(x+ht)-F(x)}{t} \\&= \lim_{t\to0}\left(2\langle\alpha x-A^*Ax+A^*y,h\rangle_X+t\|Ah\|_Y^2+t\alpha\|h\|_X^2\right) \\&= 2\langle\alpha x-A^*Ax+A^*y,h\rangle_X. \end{split} $$ The Gâteaux derivative $D_GF(x)$ is an element of $X^*$ which can be identified with $X$ via the inner product. After this identification $$ D_GF(x) = 2(\alpha x-A^*Ax+A^*y). $$ In fact, since $A$ is continuous (follows from compactness), the function $F$ is actually Fréchet differentiable and the two derivatives agree.

If $A$ were not linear, the derivative would include a derivative of $A$. I assume nonlinearity was not intended in your exercise.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.