# Doing a least squares approximation, can someone explain this step from the book?

In an example from the book, the author is finding the linear least squares approximation of $e^x$. We have a standard equation for least squares:

$$g(\alpha_0,\alpha_1) = \int_{-1}^1 [e^x-\alpha_0-\alpha_1x]^2dx$$

which turns into $$2\int_{-1}^1 [e^x-\alpha_0-\alpha_1x](-1)dx$$(which i think has something to do with the chain rule, and taking the derivative in terms of $\alpha_0$?) and

$$2\int_{-1}^1 [e^x-\alpha_0-\alpha_1x](x)dx$$ which i think corresponds to $\frac{d}{d\alpha_1}$. This is where I am confused, can someone explain exactly what happened?

I would like to reproduce this on my equation:

$$g(\alpha_0,\alpha_1) = \int_{0}^{\pi/2} [sinx-\alpha_0-\alpha_1x]^2dx$$

So knowing that my assumption was right, when doing the same to my above equation, do I end up with: $$2\int_0^{\pi/2} [sinx-\alpha_0-\alpha_1x](-1)dx=0$$ when the derivative is taken w.r.t $\alpha_0$ and:

$$2\int_0^{\pi/2} [sinx-\alpha_0-\alpha_1x](-x)dx=0$$ when $\frac{d}{d\alpha_1}$ is taken?

-

Your original function, which you are trying to minimize is

$$g(\alpha_0,\alpha_1)=\int_0^{\pi/2}[\sin(x)-\alpha_0-\alpha_1 x]^2dx$$

so what you do is you take partials of $g(\alpha_0,\alpha_1)$ with respect to all of its variables, set them all simultaneously equal to zero, and then solve for the variables. Remember that $g(\alpha_0,\alpha_1)$ is a function of only two variables, $\alpha_0$ and $\alpha_1$ so two partial derivatives and hence two equations in two variables. $x$ is not a variable here.

First,

\begin{eqnarray*} \frac{\partial g}{\partial \alpha_0}&=&\frac{\partial}{\partial \alpha_0}\left(\int_0^{\pi/2}[\sin(x)-\alpha_0-\alpha_1 x]^2dx\right)\\ &=&\int_0^{\pi/2}\frac{\partial}{\partial \alpha_0}[\sin(x)-\alpha_0-\alpha_1 x]^2dx\\ &=&\int_0^{\pi/2}(-2)[\sin(x)-\alpha_0-\alpha_1 x]dx. \end{eqnarray*}

Then,

\begin{eqnarray*} \frac{\partial g}{\partial \alpha_1}&=&\frac{\partial}{\partial \alpha_1}\left(\int_0^{\pi/2}[\sin(x)-\alpha_0-\alpha_1 x]^2dx\right)\\ &=&\int_0^{\pi/2}\frac{\partial}{\partial \alpha_1}[\sin(x)-\alpha_0-\alpha_1 x]^2dx\\ &=&\int_0^{\pi/2}(-2x)[\sin(x)-\alpha_0-\alpha_1 x]dx. \end{eqnarray*}

So you have two mistakes up above, both in the second partial. You are missing the minus sign and $x$. The second partial does NOT have $\cos x$ in it in your sixth equation.

Now solve \begin{eqnarray*} \int_0^{\pi/2}(-2)[\sin(x)-\alpha_0-\alpha_1 x]dx&=&0\\ \int_0^{\pi/2}(-2x)[\sin(x)-\alpha_0-\alpha_1 x]dx&=&0 \end{eqnarray*}

by first evaluating the integrals and getting rid of $x$. Then you have a linear 2x2 system.

BTW, your third expression is also missing a minus sign.

-
I see, this makes a lot of sense now. The book gives a high level overview of things and doesn't cover the step-by-step but it's been 4 or so years since I've done calculus and am very rusty. Thank you. – BMEdwards37 Apr 11 '13 at 13:20

You are right that the second expression results from the derivative of $g$ w/ respect to $\alpha_1$. It is correct except for a minus sign, but since you are going to set it to zero anyway, that doesn't matter.

$\frac{d}{d\alpha_1}\int [e^x - \alpha_0 -\alpha_1 x ]^2 dx= \int \frac{d}{d\alpha_1}[e^x - \alpha_0 -\alpha_1 x ]^2 dx$

Now just do the derivative and you are left with your second expression.

-
I see, that is what the professor told me and I'm still not sure 100% exactly how it works. Can you take a look at my edit and see if that seems correct? – BMEdwards37 Apr 10 '13 at 23:40
I no longer need my edit looked at but appreciate the help very much, I understand now. – BMEdwards37 Apr 11 '13 at 13:22