# Linear Algebra: Least-Squares Approximation & “Normal Equation”

I am reviewing Example 1 from Chapter 6, Section 4 (Least-Squares Approximation and Orthogonal Projection Matrices) in "Elementary Linear Algebra - A Matrix Approach 2nd Edition [ISBN] 978-0-13-187141-0"

In this example, they found a solution (2x1 matrix) of the normal equation:

 [a0]
= (((C^T)*C)^-1)(C^T)*y
[a1]


Given:

C is a 5x2 matrix; y is a 5x1 matrix

C = [1  2.60]      y = [2.00]
[1  2.72]          [2.10]
[1  2.75]          [2.10]
[1  2.67]          [2.03]
[1  2.68]          [2.04]


*((C^T)C) is a 2x2 matrix

((C^T)*C) = [5.0000   13.4200]
[13.4200  36.0322]


(C^T)*y is a 2x1 matrix

(C^T)*y = [10.2700]
[27.5743]


[a0] = [0.056]
[a1] = [0.745]


To solve this I think they had to use the formula I listed at the very top, but they did not show work for ((C^T)*C)^-1 (which I guess is the inverse). If someone can please explain with full details of how they solved this normal equation.

I at least understand the given equations I posted, but I don't know why they didn't show the steps of ((C^T)*C)^-1 and how exactly they arrived to:

[a0] = [0.056]
[a1] = [0.745]

y = 0.056 + 0.745x


The normal equation is $$C^tCx=C^ty$$ which is to say, $$\pmatrix{5&13.42\cr13.42&36.0322\cr}\pmatrix{a_0\cr a_1}=\pmatrix{10.27\cr27.5743\cr}$$ This is just solving two equations in two unknowns, and you can solve such a system by any method you know (and surely you know how to solve two equations in two unknowns).
Now, one way to solve it is to multiply both sides by $(C^tC)^{-1}$ which is, indeed, the multiplicative inverse of $C^tC$; you get the solution $x=(C^tC)^{-1}C^ty$. So your question is, how do you find the inverse of a matrix.
For $2\times2$ matrices, there is a very simple answer: $${\rm The\ inverse\ of\ }\pmatrix{a&b\cr c&d\cr}{\rm\ is\ }(ad-bc)^{-1}\pmatrix{d&-b\cr-c&a}$$