Sorry if this is a complete dummy question, but I haven't done math in years and I'm quite rusty. I'm reading this explanation of least squares regression, which internally uses the quadratic equation solution. I was wondering if someone can tell me if my analysis of this is correct. I'm trying to explain to myself why the expression is clearly minimized...

which is

$$a\left(x+\frac{b}{2a}\right)^2 + \frac{4ac-b^2}{4a}$$

Since the second term does not depend upon x, and the first term is a
times a perfect square (and a > 0), the expression is clearly
minimized when x = -b/(2a).

I suppose this is somewhat equivalent:

$z = a(x+y)^2$ + C

which we can take the derivative of to find the function for the slope of $z$:

$\frac{dx}{dz} z = 2a(x+y)$

set this equal to zero to find the point where the slope is zero (e.g. a minimum or maximum--we know that $a > 0$ so this should be a minimum)

$0 = 2a(x+y)$

$a$ is discarded during the division, and subtract to solve for $x$

$0 = x+y$

$ x =-y$


The right side is a constant, so no matter what value of $x$ we choose, it will still be there. If it was a minimum before, it will still be the minimum when we add a constant (consider a curve, then consider shifting the curve $k$ units upwards, clearly it doesn't change the minimum / maximum).

You should realize that we have the square of $x+b/2a$ on the left. The smallest the square of a real number can be is $0$ (at which point the left term will disappear and we will have our minimum). Set the $(x + b/2a)^2$ to 0 and solve to get $x = -b/2a$, which is your solution.

(Another easy way to verify that this is true by considering the derivative of $ax^2 + bx + c$ to get $2ax + b = 0$, and solve for $x = -b/2a$ as the inflection point of the quadratic.)

  • $\begingroup$ Ah, that's what I was not considering -- the constant doesn't change the minimum location, and we don't care about the value at the minimum $\endgroup$ – Hamy Mar 9 '13 at 1:52

Differentiating makes things look much more complicated than they are: all we need is some algebra. It is easy to verify (by expanding the right-hand side) that $$ax^2+bx+c=a\left(x+\frac{b}{2a}\right)^2 + \frac{4ac-b^2}{4a}$$ for all $x$. The second term is independent of $x$, it is a constant.

The term $$a\left(x+\frac{b}{2a}\right)^2$$ is the positive $a$ number times a square. A square is never negative, the smallest it can be is $0$. And in this case we can arrange for it to be $0$ by choosing $x=\frac{b}{2a}$.

It follows that the minimum possible value of $ax^2+bx+c$ is $\frac{4ac-b^2}{4a}$, and it is reached at exactly one place, namely $x=-\frac{b}{2a}$.

Remark: All these letters may hide the basic simplicity of the idea. Consider the quadratic $x^2 -6x+17$. This can be rewritten as $(x-3)^2+8$. But $(x-3)^2$ is never negative, so $(x-3)^2+8 \ge 8$ for all $x$. And $(x-3)^2 =0$ at $x=3$, so $(x-3)^2+8$ can be equal to $8$. so the minimum value of $(x-3)^2+8$ is $8$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.