Are minimizing a function and root finding the same? What is the relationship between minimizing a function and finding a root of an equation? Are the the same? I know in both problem we have similar algorithms, such as gradient decent, or newton's methods.
For example, Let use assume $x$ is a scalar. Finding a root for an equation $f(x)=b$ is checking where the function $f(x)-b$ cross the x axis. This is definitely not equal to minimize $f(x)-b$.
But in the convex optimization book, Minimizing $\|Ax-b\|_2^2$ is equal to the solution of the linear system $Ax=b$.
What I am missing? in which case we can transfer optimization to root finding?
 A: Root-finding and optimization are not the same, but a root-finding problem can be reformulated into an optimization problem. That is, we can construct an optimization problem from a root-finding problem where the solution to both problems are the same.
The value(s) of $x$ that satisfy $f(x) = b \implies f(x)-b = 0$ will also be the global minimizer(s) of the optimizition problem min $||f(x)-b||^2_2$.
The idea is that the $x^*$ that makes $f(x^*) = b$ also makes $f(x^*)-b = 0$ and will also be the global minimizer for $||f(x^*)-b||^2_2$ since the lowest value it can acheive is $0$.
*If the function $f$ and its domain have the right mathematical properties than you can convert it to a optimization problem. Things like differentiability of $f$.
A: Footnote to @Roby's answer: As a practical note, if you're using an off-the-shelf optimizer to solve $f(x)=b$ by minimizing $||f(x)-b||^2_2$, you have to be careful that a minimum you find is actually a solution. As an example, suppose you're trying to solve $f(x)=b$ for $x \in \mathbb{R}$, where
$
\begin{split}
f(x) &= x^2 + 2\\
b &= 1
\end{split}$
Then $x = 0$ minimizes $||f(x)-b||^2_2$, but it doesn't solve $f(x)=b$, since no real $x$ solves $x^2 + 2 = 1$.
So, to the question, you can use an optimization method to find roots, by minimizing $||f(x)-b||^2_2$, and checking that the objective-function value is $0$ at the minimum. That said, not all minima, even global minima, correspond to roots, since not all equations have roots.
A: Definitely not same. In your case, assuming that columns of $A$ span vector $b$ and $||Ax-b||$ being norm which is non-negative, the problem of investigating minima is similar to finding solution of $Ax-b=0$.
