Analytical solution of least square problem could anyone explain:
a) $||{Ax-b}||^2$ (there is also a lowered 2): what does this two 2's mean?
b) why is the solution: $x =(A^TA)^{-1} A^Tb$ is?
Thank you very much:)
 A: Start with the linear system 
$$
\mathbf{A}x = b,
\quad 
\mathbf{A}\in\mathbb{C}^{m\times n},
\quad
x \in \mathbf{C}^{n},
\quad
b \in \mathbf{C}^{m}.
$$
Assume $\mathbf{A}$ has full column rank where the number of columns is equal to the rank $\rho$, that is $n=\rho$. If there is no $x\in\mathcal{R}(\mathbf{A})$ that solves this equation, then instead of trying to make the residual error vector $r(x) = \mathbf{A}x - b$ be zero, make it as small as possible.
To measure size and distance, we must choose a norm. You have chosen the $2-$norm as elucidated by @Denis Düsseldorf. This is the familiar norm of Pythagoras:
$$
 \lVert x + y \rVert_{2}^{2} = x^{2} + y^{2}
$$
There are a variety of ways to solve the least squares problem. Your question is about the normal equations:
$$
\mathbf{A}^{*} \mathbf{A} x = \mathbf{A}^{*} b.
$$
By construction, this system is consistent. The vector $\mathbf{A}^{*} b$ is clearly in the range space of $\mathbf{A}^{*}$. Because we have a full column rank linear system, the $n\times n$ product matrix $\mathbf{A}^{*} \mathbf{A}$ has full rank, and is nonsingular. Therefore the inverse exists and the least squares solution is
$$
  x_{LS} = \left( \mathbf{A}^{*} \mathbf{A} \right)^{-1} \mathbf{A}^{*} b.
$$
