Approximate $f(x) = x^{1/3}$ by a polynomial $p(x)$ of degree $\le 2$ that minimizes the error $$E = (f(-1)-p(-1))^2 + \int_{-1}^1 (f(x)-p(x))^2 dx + (f(1)-p(1))^2$$

I think I must minimize the following:

$$\min ||x^{1/3}-(ax^2 + bx + c)||^2$$

but I don't know how to proceed.

  • $\begingroup$ Hint: Write E as a function of $a, b, c$. How do you find the extrema of a function of three variables? Once you have found the extrema, you have to decide which one is the minimum (if it exists). $\endgroup$
    – NickD
    Apr 13 '19 at 15:45
  • $\begingroup$ @NickD you mean $E = (f(-1)-p(-1))^2 + \int_{-1}^1 (x^{1/3}-(ax^2+bx+c))^2 dx + (f(1)-p(1))^2$? Then I should take the derivatives and equate to $0$? Do you have an idea in how to do it without using derivatives but instead use orthogonal projections? $\endgroup$ Apr 13 '19 at 17:58
  • $\begingroup$ Yes, to the first part (although you'll have to express the parts outside the integral in terms of $a, b, c$ as well). I have no idea what you mean in the second part. $\endgroup$
    – NickD
    Apr 14 '19 at 0:55
  • $\begingroup$ @NickD It is, then, $E = (-1-(a-b+c))^2 + \int_{-1}^1 (x^{1/3}-(ax^2+bx+c))^2 dx + (-1-(a+b+c))^2$ rigth? What I meant by my last phrase is that there's a way to do regression that involves projection onto things, not using the standard way of taking the derivative and equating to $0$. $\endgroup$ Apr 14 '19 at 20:13
  • $\begingroup$ Right - now you evaluate the integral and then you have an expression in $a, b, c$ that you want minimized. As for the second part, I still don't know what you mean, but this might be my ignorance speaking. $\endgroup$
    – NickD
    Apr 14 '19 at 20:29

To answer this type of questions it is vital to correctly identify the vector space, the subspace and the inner product. The error formula $$E = E(f-p) = (f(-1)-p(-1))^2 + \int_{-1}^1 (f(x)-p(x))^2 dx + (f(1)-p(1))^2$$ makes sense for all functions $f-p$ which are continuous on the interval $[-1,1]$. In particular, it makes sense when $f : [-1,1] \rightarrow \mathbb{R}$ is given by $$f(x) = x^{1/3}$$ and $p$ is any polynomial. We therefore consider the vector space $\Omega$ given by $$ \Omega = \{ f : [-1,1] \rightarrow \mathbb{R} \: : \: \text{$f$ is continuous on $[-1,1]$} \}.$$ There is no harm in stressing how addition and scalar multiplication is defined. Specifically, if $f, g \in \Omega$ and $r, s \in \mathbb{R}$, then $r\cdot f + t \cdot g$ is the function $h \in \Omega$ given by $$h(x) = r \cdot f(x) + t \cdot g(x).$$ Moreover, let $V$ be given by $$ V = \{ p : [-1,1] \rightarrow \mathbb{R} \: : \: \text{$p$ is a polynomial of degree at most $2$} \}$$ It is clear that $V$ is a subspace of $\Omega$. We now need an inner product defined all $f, g \in \Omega$. In view of the expression for $E$ we choose to define $$ \langle f, g \rangle = f(-1)g(-1) + \int_{-1}^1 f(x)g(x) dx + f(1)g(1).$$ It is straightforward but very important to verify that this definition has all the properties of an inner product. In particular, we have $$ \langle r_1 f_1 + r_2 f_2 , g \rangle = r_1 \langle f_1, g \rangle + r_2 \langle f_2 , g \rangle $$ and $$ \langle f, g \rangle = \langle g, f \rangle$$ as well as $$\langle f, f \rangle \ge 0, \quad\text{and}\quad \langle f,f \rangle = 0 \: \Leftrightarrow \forall x \in [-1,1] \: : f(x) = 0.$$ This completes our preparations. We note that $$E(f-p) = \langle f-p, f-p \rangle = \| f - p\|^2$$ is merely the square of the norm associated with our freshly defined inner product on $\Omega$. To minimize the error we merely need to project our target function $f$ onto $V$ using our inner product. To that end, we deploy Gram-Schmidt's orthogonalization procedure and construct an orthonomal basis for $V$. If $p_i \in \Omega$ is given by $$p_i(x) = x^i$$ for $i=0,1,2$, then it is clear that $$ B = \{ p_0, p_1, p_2 \} \subset V$$ is a basis for $V$, but it is almost certainly not orthonormal with respect to our inner product. In fact we have $$ \|p_0\|^2 = \langle p_0, p_0 \rangle = 1 + \int_{-1}^1 1 dx + 1 = 4$$ It follows that $v_0$ given by $$v_0 = \frac{1}{2} p_0$$ has norm $1$. Continuing with Gram-Schmidt's procedure we compute $$ \langle p_1, v_0 \rangle = (-1)\left(\frac{1}{2}\right) + \int_{-1}^1 x \left(\frac{1}{2}\right) dx + (1)\left(\frac{1}{2}\right) = 0.$$ We have been lucky! The function $p_1$ is already orthogonal to $v_0$. We compute $$ \|p_1 \|^2 = (-1)(-1) + \int_{-1}^1 x\cdot x dx + (1)(1) = 2 + \frac{2}{3} = \frac{8}{3}.$$ It follows that $v_1$ given by $$ v_1 = \sqrt{\frac{3}{8}} p_1 $$ is orthogonal to $v_0$ and has norm $1$. We now process $p_2$. We have $$ \langle p_2, v_0 \rangle = (-1)^2 \frac{1}{2} + \int_{-1}^1 x^2 \frac{1}{2} dx + (1)^2 \frac{1}{2} = 1 + \frac{2}{3} = \frac{5}{3}$$ and $$ \langle p_2, v_1 \rangle = (-1)^2 \sqrt{\frac{3}{8}} (-1) + \int_{-1}^1 x^2 \cdot \left( \sqrt{\frac{3}{8}} \right) x dx + (1)^2 \sqrt{\frac{3}{8}} (1) = 0.$$ We conclude that the intermediate polynomial $w_2$ given by $$ w_2 = p_2 - \frac{5}{3}v_0$$ is orthogonal to $v_0$ and $v_1$. Before continuing there is nothing lost by noting $$ w_2(x) = x^2 - \frac{5}{6}.$$ It remains to compute $\|w_2\|$ and define the final element $v_2 = \frac{1}{\|w_2\|}w_2$ of our new orthonormal basis $$\{v_0, v_1, v_2\}.$$ Afterwords, the orthogonal projection $q \in V$ of $f(x) = x^{1/3}$ with respect to our inner product can be computed. This polynomial $q$ will minimize the special error.

It is getting late in my timezone, so I will leave it at that for right now. It is entirely possible that I have made a miscalculation when applying GS's method. Therefore, check all the calculations carefully. However, the central idea is correct: Examine the expression for the error and find a suitable vector space with a related inner product.

  • $\begingroup$ Can you talk more about "To minimize the error we merely need to project our target function $f$ onto $V$ using our inner product"? I don't know why the projection minimizes the error. I need an explanation of this $\endgroup$ Apr 17 '19 at 22:40
  • $\begingroup$ Besides that, I completely understood your answer, and I'm very glad you helped, it's exactly what I needed. If you help me with the others it's going ot be great because I have en exam soon $\endgroup$ Apr 17 '19 at 23:16
  • $\begingroup$ I have a theory: if we analyze $||f-p||$, then if $p$ is the projection of $f$ onto the subspace generated by a basis for $p$, then $p$ is the minimizer of $||f-p||$. Why? Because $f-p$ is the vetor orthogonal to the subspace generated by $p$, and by being orthogonal the triangular inequality or pytagorean theorem says that it has the least distance between $f$ and $p$. It is easier to see with a drawing, but I think you can understand $\endgroup$ Apr 17 '19 at 23:43
  • $\begingroup$ It'd also be great if you could give me insigth about this one: math.stackexchange.com/questions/3186340/… using projections. All the answers were using calculus $\endgroup$ Apr 18 '19 at 0:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.