Least squares approximation: Legendre polynomial Find the best quadratic least squares approximation to $f(x)=e^x$ on $[-1,1]$ with respect to the inner product $\langle f(x),g(x) \rangle = \displaystyle\int_{-1}^1 f(x)g(x)dx$.
I cannot figure out how to do this. Is there a good online resource for this? Unfortunately, I don't have a book for it.
Any tips/hints would be helpful.
 A: You have to project $e^x$ on the vector space $P_2 = \{ ax^2+bx+c : a,b,c \in \mathbb{R} \}$.
Notice that a basis for $P_2$ is $1,x, x^2$. So you have to find $a,b,c \in \mathbb{R}$ such that $\langle e^x - (ax^2+bx+c), 1 \rangle = \langle e^x- (ax^2+bx+c), x \rangle = \langle e^x- (ax^2+bx+c), x^2 \rangle = 0$.
So you have three equations with three indeterminates $a,b,c$
$$
\int_{-1}^1 (e^x - (ax^2+bx+c)) \ dx=0
$$
$$
\int_{-1}^1 (e^x - (ax^2+bx+c))x \ dx=0
$$
$$
\int_{-1}^1 (e^x - (ax^2+bx+c))x^2 \ dx=0
$$
Getting
$$
\left\{
\begin{matrix}
-\frac{2}{3} a &&-2c &+e - \frac{1}{e} &=& 0 \\
 & -\frac{2}{3} b && +\frac{2}{e} &=& 0 \\
-\frac{2}{5} a && -\frac{2}{3} c &+e - \frac{5}{e} &=& 0
\end{matrix}
\right.
$$
A: Let $X$ be the inner product space in question. Note $\dim P_2 =3$. The problem is $\min_{p \in P_2} \|p-\exp\|^2$.
Note that the minimum exists by a compactness argument because $P_2$ is finite dimensional and $\|p-\exp\| \ge \|p\|-\|\exp\|$, hence has a solution $\hat{p} \in P_2$.
It follows that $(\hat{p}-\exp) \bot P_2$ (and hence that $\hat{p}$ is unique by Pythagoras).
In other words, you need to find the $\hat{p}$ that solves
$\langle \hat{p}-\exp, p \rangle = 0$ for all $p \in P_2$. 
Note that if $b_0,b_1,b_2$ is any basis of $P_2$, this reduces to
solving $\langle \hat{p}-\exp, b_k \rangle = 0$, or 
$\langle \hat{p}, b_k \rangle = \langle \exp, b_k \rangle $.
Since $\hat{p} \in P_2$, we can write $\hat{p} = \sum_i \pi_i b_i $
for some constants $\pi_i$, so the problem reduces to solving
$\sum_i \pi_i \langle b_i, b_k \rangle = \langle \exp, b_k \rangle $.
If the $b_k$ are orthogonal, the solution is immediate, otherwise this reduces to solving a $3 \times 3$ system of equations.
The Legendre polynomials $P_k$ are orthogonal, hence letting $b_k =P_k$, we obtain
$\pi_k = { \langle \exp, P_k \rangle \over \|P_k\|^2 }$.
