Least squares problem: find the line through the origin in $\mathbb{R}^{3}$ The problem is as follows:
"Please set up (but do not solve) the normal equations for the following least squares
approximation problem: Find $(a, b, c, d)$ such that the plane H described by $ax + by + cz = d$
minimizes $\sum |ax_i + by_i + cz_i − d|^2$ where the $(x1, y1, z1), \cdots ,(x6, y6, z6)$ are the following points: $(2,3,4)$, $(99,−85,0)$, $(0,1,8)$, $(5,2,2)$, $(3,3,3)$, $(1,2,4)$."
I solved a similar problem with points representing ordered pairs, for example $(2,1)$, $(3,5)$ and so on, but the question was to get a line that approximates such ordered pairs. In that case I represented the points as $y = mx + b$ and got $m$ and $b$ for the new line. I wonder if for the new problem which is a 3D space I also have to represent the ordered triplets as $z = ax + by + c$ or something like that.
I will very much appreciate any clue.
 A: You are on the right track. Set up your equations like you would in 2 dimensions.
\begin{align*}
z_1 &= d + ax_1 + by_1 \\
z_2 &= d + ax_2 + by_2 \\
&\vdots \\
z_6 &= d + ax_6 + by_6.
\end{align*}
From this, you can hopefully derive the normal equations as matrix products.
A: We have our data points $\left\{ x_{k}, y_{k}, z_{k}, d_{k} \right\}_{k=1}^{m}$, and our trial function, a line in $\mathbb{R}^{3}$ through the origin:
$$
 d\left( x, y, z \right) = a x + b y + c z.
$$
The linear system is
$$
\begin{align}
  \mathbf{A} a &= d \\
%
  \left[ \begin{array}{ccc}
    x & y & z 
  \end{array} \right]
%
\left[ \begin{array}{c}
 a
\end{array} \right]
%
 & =
%
 \left[ \begin{array}{c}
   d
 \end{array} \right]\\[3pt]  
%
\left[ \begin{array}{ccc}
  x_{1} & y_{1} & z_{1} \\
  x_{2} & y_{2} & z_{2} \\
  \vdots & \vdots & \vdots \\
  x_{m} & y_{m} & z_{m} \\
\end{array} \right]
%
\left[ \begin{array}{c}
 a \\ b \\ c
\end{array} \right]
%
&=
%
\left[ \begin{array}{c}
 d_{1} \\ d_{2} \\ \vdots \\ d_{m} 
\end{array} \right].
%
\end{align}
$$
The normal equations are
$$
  \begin{align}
   \mathbf{A}^{*} \mathbf{A} a &= \mathbf{A}^{*} d \\
\left[
  \begin{array}{ccc}
    x \cdot x & x \cdot y & z \cdot z \\
    y \cdot x & y \cdot y & y \cdot z \\
    x \cdot z & x \cdot z & z \cdot z \\
  \end{array}
\right]
%
\left[
  \begin{array}{c}
    a \\
    b \\
    c
  \end{array}
\right]
%
 &=
%
\left[
  \begin{array}{c}
    x \cdot d \\
    y \cdot d \\
    z \cdot d
  \end{array}
\right].
%
  \end{align}
%
$$
The solution is
$$
  a_{LS} =
%
\left[
  \begin{array}{c}
    a \\
    b \\
    c
  \end{array}
\right]_{LS}
%
= \left( \mathbf{A}^{*}\mathbf{A} \right)^{-1} \mathbf{A}^{*} d.
$$
