I am reviewing Example 1 from Chapter 6, Section 4 (Least-Squares Approximation and Orthogonal Projection Matrices) in "Elementary Linear Algebra - A Matrix Approach 2nd Edition [ISBN] 978-0-13-187141-0"

In this example, they found a solution (2x1 matrix) of the normal equation:

        = (((C^T)*C)^-1)(C^T)*y


C is a 5x2 matrix; y is a 5x1 matrix

C = [1  2.60]      y = [2.00]
    [1  2.72]          [2.10]
    [1  2.75]          [2.10]
    [1  2.67]          [2.03]
    [1  2.68]          [2.04]

*((C^T)C) is a 2x2 matrix

((C^T)*C) = [5.0000   13.4200]
            [13.4200  36.0322]

(C^T)*y is a 2x1 matrix

(C^T)*y = [10.2700]

The answer was:

[a0] = [0.056]
[a1] = [0.745]

To solve this I think they had to use the formula I listed at the very top, but they did not show work for ((C^T)*C)^-1 (which I guess is the inverse). If someone can please explain with full details of how they solved this normal equation.

I at least understand the given equations I posted, but I don't know why they didn't show the steps of ((C^T)*C)^-1 and how exactly they arrived to:

[a0] = [0.056]
[a1] = [0.745] 

 y = 0.056 + 0.745x

The normal equation is $$C^tCx=C^ty$$ which is to say, $$\pmatrix{5&13.42\cr13.42&36.0322\cr}\pmatrix{a_0\cr a_1}=\pmatrix{10.27\cr27.5743\cr}$$ This is just solving two equations in two unknowns, and you can solve such a system by any method you know (and surely you know how to solve two equations in two unknowns).

Now, one way to solve it is to multiply both sides by $(C^tC)^{-1}$ which is, indeed, the multiplicative inverse of $C^tC$; you get the solution $x=(C^tC)^{-1}C^ty$. So your question is, how do you find the inverse of a matrix.

For $2\times2$ matrices, there is a very simple answer: $${\rm The\ inverse\ of\ }\pmatrix{a&b\cr c&d\cr}{\rm\ is\ }(ad-bc)^{-1}\pmatrix{d&-b\cr-c&a}$$

For bigger matrices, there is a simple procedure, involving row reduction.

But surely all of this is in some earlier chapter of the textbook you're using?

  • $\begingroup$ I totally forgot the inverse of a 2x2! Thank you! XD $\endgroup$ – Sol Bethany Reaves May 1 '13 at 12:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.