In De Boor (1972) it is stated that

Let be $\$ $ a finite dimensional linear space of functions defined on the interval $[a,b]$. We are searching for the best approximation from $\$$ to $g$.

The function $f^*$ is a best approximation from $\$$ to $g$ with respect to the $\mathcal{L}_2$ norm if and only if the function $f^*$ is in $\$$ and the error term $g-f^*$ is orthogonal to $\$$.

To show that the double implication holds the author provides the following statement (I add my own procedure because it's not given in the book).

For any function $f\in \$\ $ we have that $$\|g-f\|_2^2=\|g-f^*+f^*-f\|_2^2=\|g-f^*\|_2^2+2\langle f^*-f,g-f^*\rangle+\|f^*-f\|_2^2$$

If condition is satisfied we have

$$\|g-f\|_2^2=\|g-f^*\|+\|f^*-f\|_2^2\geq\|g-f^*\|_2^2$$ Which proves that $f^*$ is the best least squares approximation in $\$$

If $\langle f,g-f^*\rangle \neq 0$ we have that by letting $tf:=f^*-f$

$$\|g-f\|_2^2=\|g-(f^*+tf)+(f^*+tf)-f\|_2^2$$ $$=\|g-(f^*+tf)\|_2^2+2\langle 2tf,g-f^*-tf\rangle+\|2tf\|_2^2$$

$$=\|g-(f^*+tf)\|_2^2+4\langle tf,g-f^*\rangle$$ Given that for all nonzero $t$ of the same sign as $\langle f,g-f^*\rangle$ and sufficiently close to $0$.

$$2t\langle f,g-f^*\rangle>\|tf\|_2^2$$ This completes the proof since it implies that \begin{equation} \|g-(f^*+tf)\|_2^2<\|g-f^*\|_2^2 \end{equation} and hence $f^*$ is not the best approximation.

I am not sure to understand the reason why it holds that $$2t\langle f,g-f^*\rangle>\|tf\|_2^2$$

Is it because $t^2$ goes to zero faster than $t$. Is there some way to show it more formally? Thanks in advance.

  • $\begingroup$ I suspect there are some typos, e.g. $\langle f^*-f,g-f^*\rangle$ in the first line after "we have that" and $||g-f||_2^2=||g-f^*||_2^2+||f^*-f||_2^2\geq ||g-f^*||_2^2$ $\endgroup$ – Peter Melech Jan 23 at 16:33
  • $\begingroup$ @PeterMelech I agree with you about the second typo, but the first one,( the inner product) looks okey to me, what do you think? $\endgroup$ – RScrlli Jan 23 at 16:37
  • 1
    $\begingroup$ I'm quite sure it has to be $f^*$ instead of $f$ in the right argument of the inner product, similarly: If $\langle f,g-f^*\rangle\neq 0$ some lines under, when showing the other direction $\endgroup$ – Peter Melech Jan 23 at 16:46
  • $\begingroup$ Besides: wonder how" by letting $tf:=f^*-f$" is to be understood $\endgroup$ – Peter Melech Jan 23 at 16:57
  • $\begingroup$ @PeterMelech you are right, I've done a copy and paste from another document and I didn't realize that. Thanks! $\endgroup$ – RScrlli Jan 23 at 16:57

I would like to suggest a slightly different proof, because I don't feel well with the argument " by letting $tf:=f^*-f$".

Assume $\langle f,g-f^*\rangle\neq 0$ for some $f\in U$ (I call the subspace $U$), then for all $t\in\mathbb{R}$ one has: $$||g-f^*||_2^2=||g-f^*+tf-tf||_2^2=||g-(f^*+tf)||_2^2+2\langle g-(f^*+tf),tf\rangle +||tf||_2^2$$ $$=||g-(f^*+tf)||_2^2+2t\langle g-f^*,f\rangle -2t^2\langle f,f\rangle+||tf||_2 ^2$$ $$=|||g-(f^*+tf)||_2^2+2t\langle g-f^*,f\rangle-t^2||f||_2^2.$$ And now choose $t$ so that $$2t\langle g-f^*,f\rangle-t^2||f||_2^2>0$$ which in case that $t$ has the same sign as $\langle g-f^*,f\rangle$ is equivalent to $$|t|<\frac{2|\langle g-f^*,f\rangle|}{||f||_2^2}.$$ (Note that $\langle g-f^*,f\rangle\neq 0$ is essential to that.) Then: $$|||g-(f^*+tf)||_2^2<||g-f^*||_2^2$$ which contradicts the best-approximation-property.

  • $\begingroup$ It's much more clear and general as a proof, for sure I will include it in my thesis with a footnote thanking you for the hint $\endgroup$ – RScrlli Jan 24 at 15:27
  • $\begingroup$ Thank You very much! $\endgroup$ – Peter Melech Jan 25 at 7:49

If $t$ has same sign as $\langle f, g-f^{*}\rangle$ the inequality you want is $2|t||\langle f, g-f^{*}\rangle| >|t|^{2} \|f\|_2^{2}$. This is true whenever $|t| <\frac {2|\langle f, g-f^{*}\rangle|} { \|f\|_2^{2}}$.

  • $\begingroup$ by taking the absolute value everything makes sense, but the problem is that what I want to show is $2\langle f^*-f,g-f\rangle+\|f^*-f\|_2^2>4\langle tf,g-f^*\rangle$. $\endgroup$ – RScrlli Jan 23 at 10:40
  • $\begingroup$ I've edited the question, showing the first part of the proof. $\endgroup$ – RScrlli Jan 23 at 10:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.