Least square fit using Legendre polynomials

I would like to apply Legendre polynomials to least square approximation. Therefore I would like the function:

$$L_n (x)=\sum_{k=0}^n a_k P_k (x)$$

to fit $f(x)$ defined over $[-1,1]$ in a least square sense.

We should minimize:

$$I(a_0, ..., a_n)= \int_{-1}^1 [f(x) - L_n (x)]^2 \; dx\tag1$$

and so we must set

$$\frac{\partial I}{\partial a_r} = 0,\qquad r=0,1, \ldots,n\tag2$$

Using equations $(1)$ and $(2)$

$$\int_{-1}^1 P_r(x) \left[f(x) - \sum_{k=0}^n a_k P_k (x)\right]dx = 0,\qquad r=0,1, \ldots,n$$

should be an equivalent term.

My question now is: why is that true?

I would be glad if someone could illustrate the last step with more details. Thanks, Rainier.

-
thx for editing, looks much nicer now. – rainier Sep 15 '12 at 15:58
It is true because the functional $I(a_0\ldots a_n)$ that you want to minimize is convex. Hence any critical point is a minimum. – Giuseppe Negro Sep 15 '12 at 16:10
ok, equation (1) and (2) make perfect sense to me. The step which I don't understand is how to get to the last term. – rainier Sep 15 '12 at 16:32

You have

$$\frac{\partial}{\partial a_r}\int_{-1}^1 [f(x) - L_n (x)]^2dx=0\\ \int_{-1}^1 \frac{\partial}{\partial a_r}[f(x) - L_n (x)]^2dx=0\\ \int_{-1}^1 2[f(x) - L_n (x)]\frac{\partial}{\partial a_r}[f(x) - L_n (x)]dx=0\\ \int_{-1}^1 2[f(x) - L_n (x)][0 - \frac{\partial}{\partial a_r}L_n (x)]dx=0\\ -\int_{-1}^1 2[f(x) - L_n (x)]\frac{\partial}{\partial a_r}\sum a_kP_k(x)dx=0\\ -\int_{-1}^1 2[f(x) - L_n (x)]P_r(x)dx=0$$

-

It is even easier than that. Because the Legendre polynomials are orthogonal, you can get the coefficients just from $a_n=\frac {2n+1}2\int_{-1}^1f(x)P_n(x)dx$

-

This is coming late but for the benefit of others, let me post my opinion:

$$\frac{\partial\sum_{k=0}^n a_k P_k (x)}{\partial a_r} = \frac{\partial}{\partial a_r} [a_0 P_0 + a_1 P_1 + ... + a_r P_r + ... + a_k P_k] = P_r (x)$$

This is because all other coefficients of $$P_k (x)$$ are treated as constants except for when $$k=r$$

The Orthogonality of the Legendre Polynomial was what was responsible for the final step

$$\int_{-1}^1 P_r(x)P_k (x)dx = 0, \qquad r=0,1, \ldots,n$$

The above equation is true as long as r is not equal to k

-