At a minimum or a maximum why does the first approximation make no difference with small variations? In an ordinary function like the temperature—one of the properties of the minimum is that if we go away from the minimum in the first order, the deviation of the function from its minimum value is only second order.
At any place else on the curve, if we move a small distance the value of the function changes also in the first order. But at a minimum, a tiny motion away makes, in the first approximation, no difference.
Can anyone explain geometrically , algebraically or otherwise why this is true?
For the background of where this comes from in case of any confusion the history can be easily found in the first short paragraph below figure 19-7   provided by this link http://www.feynmanlectures.caltech.edu/II_19.html#Ch19-SUM
( Not sure how to cut paste pictures yet, but figure 19-8 is one click away. ) The other caveat is that this question may be more appropriate for a physicist. 
 A: We say that a function $f: \mathbb{R} \rightarrowtail \mathbb{R}$ is differentiable at a point $a \in \text{Int}(\text{dom}(f))$ if there exists $A \in \mathbb{R}$ and $r: \text{dom}(f) \to \mathbb{R}$ function with $\lim\limits_{x \to a} \frac{r(x)}{x-a}$ so that
$$f(x)=f(a)+A(x-a)+r(x)$$
And of course $A=f'(a)$.
So if we have that $f'(a)=0$, then
$$f(x)=f(a)+r(x)$$
So there is no first order change in $f$.  
An example: $f(x):=x^2$ and $a:=0$. Then we have that
$$f(x)=f(0)+f'(0)(x-0)+r(x)$$
Substituting back everything:
$$x^2=0+0*x+r(x)$$
$$r(x)=x^2$$
So as you can see, the change around $a=0$ is second order. On the other hand, if we pick a different $a$, for example $a:=1$, we get that
$$f(x)=f(1)+f'(1)(x-1)+r(x)$$
$$f(x)=1+2(x-1)+r(x)$$
So we have a first order term as well. And the higher order term will be:
$$r(x)=x^2-1-2(x-1)$$
$$r(x)=x^2-2x+1$$
$$r(x)=(x-1)^2$$
I used here that I already know $f'(a)$, but you can calculate it this way:
$$f(x)-f(a)=A(x-a)+r(x)$$
$$\frac{f(x)-f(a)}{x-a}=A+\frac{r(x)}{x-a}$$
And now you can let $x \to a$, use the properties of $r$, and you will get that $A$ is just $f'(a)$.
A: Assuming a sufficiently smooth function, the Taylor expansion around a point $\,a\,$ can be written as:
$$
f(x) = f(a) + {\frac {f'(a)}{1!}}(x-a)+{\frac {f''(a)}{2!}}(x-a)^{2}+ \mathcal{O}\big((x-a)^3\big)
$$
If $\,a\,$ is a local extremum then $\,f'(a)=0\,$, so the lowest non-constant term is of $2^{nd}$ order or higher.
A: Here's a geometric reason. The first approximation to a differentiable function at a point follows the tangent line instead of the curve as you move away from the point. At a minimum or maximum the tangent is horizontal. The first approximation is $0$ change; you have to look at the second approximation to get an idea of how the graph departs from horizontal.
