# Proving that a given function $f^*$ is the best least square approximation

In De Boor (1972) it is stated that

Let be $$\$$ a finite dimensional linear space of functions defined on the interval $$[a,b]$$. We are searching for the best approximation from $$\$$ to $$g$$.

The function $$f^*$$ is a best approximation from $$\$$ to $$g$$ with respect to the $$\mathcal{L}_2$$ norm if and only if the function $$f^*$$ is in $$\$$ and the error term $$g-f^*$$ is orthogonal to $$\$$.

To show that the double implication holds the author provides the following statement (I add my own procedure because it's not given in the book).

For any function $$f\in \\$$ we have that $$\|g-f\|_2^2=\|g-f^*+f^*-f\|_2^2=\|g-f^*\|_2^2+2\langle f^*-f,g-f^*\rangle+\|f^*-f\|_2^2$$

If condition is satisfied we have

$$\|g-f\|_2^2=\|g-f^*\|+\|f^*-f\|_2^2\geq\|g-f^*\|_2^2$$ Which proves that $$f^*$$ is the best least squares approximation in $$\$$

If $$\langle f,g-f^*\rangle \neq 0$$ we have that by letting $$tf:=f^*-f$$

$$\|g-f\|_2^2=\|g-(f^*+tf)+(f^*+tf)-f\|_2^2$$ $$=\|g-(f^*+tf)\|_2^2+2\langle 2tf,g-f^*-tf\rangle+\|2tf\|_2^2$$

$$=\|g-(f^*+tf)\|_2^2+4\langle tf,g-f^*\rangle$$ Given that for all nonzero $$t$$ of the same sign as $$\langle f,g-f^*\rangle$$ and sufficiently close to $$0$$.

$$2t\langle f,g-f^*\rangle>\|tf\|_2^2$$ This completes the proof since it implies that $$$$\|g-(f^*+tf)\|_2^2<\|g-f^*\|_2^2$$$$ and hence $$f^*$$ is not the best approximation.

I am not sure to understand the reason why it holds that $$2t\langle f,g-f^*\rangle>\|tf\|_2^2$$

Is it because $$t^2$$ goes to zero faster than $$t$$. Is there some way to show it more formally? Thanks in advance.

• I suspect there are some typos, e.g. $\langle f^*-f,g-f^*\rangle$ in the first line after "we have that" and $||g-f||_2^2=||g-f^*||_2^2+||f^*-f||_2^2\geq ||g-f^*||_2^2$ – Peter Melech Jan 23 at 16:33
• @PeterMelech I agree with you about the second typo, but the first one,( the inner product) looks okey to me, what do you think? – RScrlli Jan 23 at 16:37
• I'm quite sure it has to be $f^*$ instead of $f$ in the right argument of the inner product, similarly: If $\langle f,g-f^*\rangle\neq 0$ some lines under, when showing the other direction – Peter Melech Jan 23 at 16:46
• Besides: wonder how" by letting $tf:=f^*-f$" is to be understood – Peter Melech Jan 23 at 16:57
• @PeterMelech you are right, I've done a copy and paste from another document and I didn't realize that. Thanks! – RScrlli Jan 23 at 16:57

I would like to suggest a slightly different proof, because I don't feel well with the argument " by letting $$tf:=f^*-f$$".
Assume $$\langle f,g-f^*\rangle\neq 0$$ for some $$f\in U$$ (I call the subspace $$U$$), then for all $$t\in\mathbb{R}$$ one has: $$||g-f^*||_2^2=||g-f^*+tf-tf||_2^2=||g-(f^*+tf)||_2^2+2\langle g-(f^*+tf),tf\rangle +||tf||_2^2$$ $$=||g-(f^*+tf)||_2^2+2t\langle g-f^*,f\rangle -2t^2\langle f,f\rangle+||tf||_2 ^2$$ $$=|||g-(f^*+tf)||_2^2+2t\langle g-f^*,f\rangle-t^2||f||_2^2.$$ And now choose $$t$$ so that $$2t\langle g-f^*,f\rangle-t^2||f||_2^2>0$$ which in case that $$t$$ has the same sign as $$\langle g-f^*,f\rangle$$ is equivalent to $$|t|<\frac{2|\langle g-f^*,f\rangle|}{||f||_2^2}.$$ (Note that $$\langle g-f^*,f\rangle\neq 0$$ is essential to that.) Then: $$|||g-(f^*+tf)||_2^2<||g-f^*||_2^2$$ which contradicts the best-approximation-property.
If $$t$$ has same sign as $$\langle f, g-f^{*}\rangle$$ the inequality you want is $$2|t||\langle f, g-f^{*}\rangle| >|t|^{2} \|f\|_2^{2}$$. This is true whenever $$|t| <\frac {2|\langle f, g-f^{*}\rangle|} { \|f\|_2^{2}}$$.
• by taking the absolute value everything makes sense, but the problem is that what I want to show is $2\langle f^*-f,g-f\rangle+\|f^*-f\|_2^2>4\langle tf,g-f^*\rangle$. – RScrlli Jan 23 at 10:40