if we are given a set of points $\{(x_i,y_i)\}$ and we are looking to fit a straight line $ax+b$ "as close" to the points as possible we are building a set of equations.

One way is to take the partial derivatives with respect to $a,b$ and find the min.

On the matrix form we multiply by $A^T$ when $A$ is entries of $1,x,x^2,..$ why is it smilier to finding the min using derivative?

  • $\begingroup$ Smilier ? What does it mean? $\endgroup$ – mathreadler Jan 14 at 10:52
  • 1
    $\begingroup$ Similar, I guess ? $\endgroup$ – nicomezi Jan 14 at 10:55

There is some cookbook about matrix and vector calculus on the internet where we can derive $${\bf x_o}=\min_{\bf x}\{\|{\bf Ax-b}\|_2^2\}$$ by

  1. Differentiating the expression $$\|{\bf Ax-b}\|_2^2=({\bf Ax-b})^T({\bf Ax-b})$$ (With respect to $\bf x$)
  2. Setting this derivative to zero.
  3. Solving for $\bf x$

Doing these steps is a good exercise.

  • $\begingroup$ Wow this great, must be bookmarked. $\endgroup$ – newhere Jan 14 at 15:38
  • $\begingroup$ I need to derive $(Ax-b)^T(Ax-b)$ is deriving matrices-by-vectors? $\endgroup$ – newhere Jan 14 at 15:54
  • 1
    $\begingroup$ You can multiply it together and differentiate each resulting term with help of the book. $\endgroup$ – mathreadler Jan 14 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.