Finding the least squares solution for the system of equations $y=Ax^2+B$ Find the least squares solution for the system of equations $y = Ax^2 + B$ where
$(x, y)$ belongs to the set {$(0, 1),(1, 5),(−1, 3)$}. What is the geometric (graphical) interpretation
of the solution? 
Is this done the same way as a least squares solution for $Ax=B$? If not then how do I start this?
 A: It's still a linear least squares problem.  If it helps, think of $x^2$ as a new variable $z$, so your equation is $y = A z + B$ and your data points are $(z,y) = (0^2,1), (1^2,5), ((-1)^2,3)$.
A: Constant + quadratic
The trial function is 
$$y(x) = \alpha + \gamma x^2.
$$
The input vectors are 
$$
x = \left[
    \begin{array}{r}
      0\\
      1\\
     -1
    \end{array}
\right], \quad 
y = \left[
    \begin{array}{r}
      1\\
      5\\
      3
    \end{array}
\right]
$$
They imply the linear system of column vectors:
$$
\begin{align}
 \mathbf{A} c &= y \\
  \left[
    \begin{array}{rr}
      1 & x^2
    \end{array}
    \right]
    \left[
    \begin{array}{r}
      \alpha \\
      \gamma
    \end{array}
    \right]
      & =
    \left[
    \begin{array}{rr}
      y
    \end{array}
    \right]
\end{align}
$$
The system matrix $\mathbf{A}$ is built with the $x$ data, the data vector is constructed with the $y$ data. The task is to find the coefficient vector $c$.
Using the input data, we have
$$
\begin{align}
 \mathbf{A} c &= y \\
\left[
    \begin{array}{rr}
      1 & 0\\
      1 & 1\\
      1 & 1
    \end{array}
\right]
\left[
    \begin{array}{r}
      \alpha \\
      \gamma
    \end{array}
\right]
&  =
\left[
    \begin{array}{r}
      1\\
      5\\
      3
    \end{array}
\right]
\end{align}
$$
Use the normal equations
$$
\mathbf{A}^{*}\mathbf{A} c =\mathbf{A}^{*}y
$$
 to find the least squares solution:
$$
c = \left( \mathbf{A}^{*}\mathbf{A} \right)^{-1}\mathbf{A}^{*}y
$$
The product matrix and its inverse are
$$
\mathbf{A}^{*}\mathbf{A} =
%
    \left[
    \begin{array}{rr}
      3 & 2\\
      2 & 2
    \end{array}
    \right], \qquad
%
\left( \mathbf{A}^{*}\mathbf{A} \right)^{-1} =
%
\left[
    \begin{array}{rr}
      1 & -1\\
     -1 & \frac{3}{2}
    \end{array}
\right].
$$
The solution vector is 
$$
c = \left[
    \begin{array}{r}
      \alpha\\
      \gamma
    \end{array}
\right]
 = 
\left[
    \begin{array}{r}
      1\\
      3
    \end{array}
\right],
$$ 
that is, the solution function is $y(x) = 1 + 3x^2$. The error vector is
$$
  r = \mathbf{A} c - y =
\left[
    \begin{array}{r}
       0 \\
      -1 \\
       1
    \end{array}
\right], \qquad
r \cdot r = 2.
$$
The solution curve is plotted against the data points below.

Constant + linear
With the quadratic fit complete, turn to the linear fit. The trial function becomes 
$$
y(x) = \alpha + \beta x
$$
The $x$ and $y$ vectors are unchanged. The linear system of column vectors is
$$
\begin{align}
 \mathbf{A} c &= y \\
\left[
    \begin{array}{rr}
      1 & 0\\
      1 & 1\\
      1 & -1
    \end{array}
\right]
\left[
    \begin{array}{r}
      \alpha \\
      \beta
    \end{array}
\right]
&  =
\left[
    \begin{array}{r}
      1\\
      5\\
      3
    \end{array}
\right]
\end{align}
$$
With the input data, we have
$$ \left[
    \begin{array}{rr}
      1 & 0\\
      1 & 1\\
      1 & -1
    \end{array}
\right]
\left[
    \begin{array}{r}
      \alpha \\
      \beta
    \end{array}
\right]
  =
\left[
    \begin{array}{r}
      1\\
      5\\
      3
    \end{array}
\right]
$$
The product matrix and its inverse are
$$
\mathbf{A}^{*}\mathbf{A} =
%
    \left[
    \begin{array}{rr}
      3 & 0 \\
      0 & 2
    \end{array}
    \right], \qquad
\left( \mathbf{A}^{*}\mathbf{A} \right)^{-1} = \frac{1}{6}
\left[
    \begin{array}{rr}
      2 & 0 \\
      0 & 3
    \end{array}
\right].
$$
The solution vector here is 
$$c = 
\left[
    \begin{array}{r}
      3\\
      1
    \end{array}
\right],
$$
so the solution function is $y(x) = 3 + x$. The error vector is
$$
  r = \mathbf{A} c - y =
\left[
    \begin{array}{r}
       2 \\
      -1 \\
       1
    \end{array}
\right], \qquad
r \cdot r = 6.
$$
The first solution represented a quadratic equation, the second a linear equation (see plot below).

To emphasize the difference between the quadratic solution $y_{q}$ and the linear solution $y_{l}$, they are written with terms in increasing order:
$$
        \begin{matrix}
        y_{q}(x) & = & \alpha_q &   &         & + & \gamma x^2\\
        y_{l}(x) & = & \alpha_l & + & \beta x
        \end{matrix}
$$
Constant + linear + quadratic
The final case is
$$
  y(x) = \alpha + \beta x + \gamma x^{2}.
$$
The linear system is
$$
\begin{align}
\mathbf{A} c &= y \\
 \left[
    \begin{array}{rr}
      1 &  0 & 0\\
      1 &  1 & 1\\
      1 & -1 & 1
    \end{array}
\right]
\left[
    \begin{array}{r}
      \alpha \\
      \beta \\
      \gamma
    \end{array}
\right]
 & =
\left[
    \begin{array}{r}
      1\\
      5\\
      3
    \end{array}
\right]
\end{align}
$$
The product matrix and its inverse are
$$
\mathbf{A}^{*}\mathbf{A} =
%
\left[
\begin{array}{ccc}
 3 & 0 & 2 \\
 0 & 2 & 0 \\
 2 & 0 & 2 \\
\end{array}
\right], \qquad
\left( \mathbf{A}^{*}\mathbf{A} \right)^{-1} = \frac{1}{2}
\left[
\begin{array}{rrr}
 2 & 0 & -2 \\
 0 & 1 & 0 \\
 -2 & 0 & 3 \\
\end{array}
\right].
$$
The solution vector is: 
$$c = 
\left[
    \begin{array}{r}
      1\\
      1\\
      3
    \end{array}
\right].
$$
The solution vector is $y(x)  = 1 + x + 3x^{2}$. The error vector is
$$
  r = \mathbf{A} c - y =
\left[
    \begin{array}{r}
       0 \\
       0 \\
       0
    \end{array}
\right], \qquad
r \cdot r = 0.
$$

