Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

In an example from the book, the author is finding the linear least squares approximation of $e^x$. We have a standard equation for least squares:

$$g(\alpha_0,\alpha_1) = \int_{-1}^1 [e^x-\alpha_0-\alpha_1x]^2dx$$

which turns into $$2\int_{-1}^1 [e^x-\alpha_0-\alpha_1x](-1)dx$$(which i think has something to do with the chain rule, and taking the derivative in terms of $\alpha_0$?) and

$$2\int_{-1}^1 [e^x-\alpha_0-\alpha_1x](x)dx$$ which i think corresponds to $\frac{d}{d\alpha_1}$. This is where I am confused, can someone explain exactly what happened?

I would like to reproduce this on my equation:

$$g(\alpha_0,\alpha_1) = \int_{0}^{\pi/2} [sinx-\alpha_0-\alpha_1x]^2dx$$

So knowing that my assumption was right, when doing the same to my above equation, do I end up with: $$2\int_0^{\pi/2} [sinx-\alpha_0-\alpha_1x](-1)dx=0$$ when the derivative is taken w.r.t $\alpha_0$ and:

$$2\int_0^{\pi/2} [sinx-\alpha_0-\alpha_1x](-x)dx=0$$ when $\frac{d}{d\alpha_1}$ is taken?

share|improve this question

2 Answers 2

up vote 1 down vote accepted

Your original function, which you are trying to minimize is

$$g(\alpha_0,\alpha_1)=\int_0^{\pi/2}[\sin(x)-\alpha_0-\alpha_1 x]^2dx$$

so what you do is you take partials of $g(\alpha_0,\alpha_1)$ with respect to all of its variables, set them all simultaneously equal to zero, and then solve for the variables. Remember that $g(\alpha_0,\alpha_1)$ is a function of only two variables, $\alpha_0$ and $\alpha_1$ so two partial derivatives and hence two equations in two variables. $x$ is not a variable here.

First,

\begin{eqnarray*} \frac{\partial g}{\partial \alpha_0}&=&\frac{\partial}{\partial \alpha_0}\left(\int_0^{\pi/2}[\sin(x)-\alpha_0-\alpha_1 x]^2dx\right)\\ &=&\int_0^{\pi/2}\frac{\partial}{\partial \alpha_0}[\sin(x)-\alpha_0-\alpha_1 x]^2dx\\ &=&\int_0^{\pi/2}(-2)[\sin(x)-\alpha_0-\alpha_1 x]dx. \end{eqnarray*}

Then,

\begin{eqnarray*} \frac{\partial g}{\partial \alpha_1}&=&\frac{\partial}{\partial \alpha_1}\left(\int_0^{\pi/2}[\sin(x)-\alpha_0-\alpha_1 x]^2dx\right)\\ &=&\int_0^{\pi/2}\frac{\partial}{\partial \alpha_1}[\sin(x)-\alpha_0-\alpha_1 x]^2dx\\ &=&\int_0^{\pi/2}(-2x)[\sin(x)-\alpha_0-\alpha_1 x]dx. \end{eqnarray*}

So you have two mistakes up above, both in the second partial. You are missing the minus sign and $x$. The second partial does NOT have $\cos x$ in it in your sixth equation.

Now solve \begin{eqnarray*} \int_0^{\pi/2}(-2)[\sin(x)-\alpha_0-\alpha_1 x]dx&=&0\\ \int_0^{\pi/2}(-2x)[\sin(x)-\alpha_0-\alpha_1 x]dx&=&0 \end{eqnarray*}

by first evaluating the integrals and getting rid of $x$. Then you have a linear 2x2 system.

BTW, your third expression is also missing a minus sign.

share|improve this answer
    
I see, this makes a lot of sense now. The book gives a high level overview of things and doesn't cover the step-by-step but it's been 4 or so years since I've done calculus and am very rusty. Thank you. –  BMEdwards37 Apr 11 '13 at 13:20

You are right that the second expression results from the derivative of $g$ w/ respect to $\alpha_1$. It is correct except for a minus sign, but since you are going to set it to zero anyway, that doesn't matter.

$\frac{d}{d\alpha_1}\int [e^x - \alpha_0 -\alpha_1 x ]^2 dx= \int \frac{d}{d\alpha_1}[e^x - \alpha_0 -\alpha_1 x ]^2 dx$

Now just do the derivative and you are left with your second expression.

share|improve this answer
    
I see, that is what the professor told me and I'm still not sure 100% exactly how it works. Can you take a look at my edit and see if that seems correct? –  BMEdwards37 Apr 10 '13 at 23:40
    
I no longer need my edit looked at but appreciate the help very much, I understand now. –  BMEdwards37 Apr 11 '13 at 13:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.