In Nocedal and Wright's book on Numerical Optimization. On page 14 we have theorem 2.1 - Taylors theorem, equation 2.6 which is, I think an approximation for a multi variable function f which is supposed to be twice continuously differentiable.

enter image description here

From the answers to this question I understand how 2.4 and 2.5 was derived. But I don't see how we can get 2.6. It looks like 2.5 was integrated to get 2.6 but it's not clear what the limits would be. Also I think it should be an approximation there in 2.6 since it's only the taylor's expansion to 2nd order terms.

if $p$ was a vector of small magnitude, to cause a perturbation around $x_o$ for the function $f$, I'd imagine the equation should look like this,

my ans

So instead of $x+tp$ in the 3rd term, I would expect just $x$. Also if it's an equality sign between LHS and RHS, I would need to mention the extra terms of 3rd order and beyond.

So is 2.6 correct, if it is how can I derive it?

  • $\begingroup$ Note that these are not approximations, they are equalities. So there are no "leftover small terms". The tradeoff is that the third term involves evaluating $f$ at a point which is probably different from $x$. $\endgroup$
    – MPW
    Aug 22, 2019 at 17:39
  • $\begingroup$ Yeah they evaluate f at $x+tp$ for "some" t, which I think means that you can find a t for which the equality is satisfied? $\endgroup$
    – Aditya P
    Aug 22, 2019 at 18:07
  • 2
    $\begingroup$ Yes, precisely. It’s an existence statement. $\endgroup$
    – MPW
    Aug 22, 2019 at 18:35

1 Answer 1


By the Taylor approximation in one dimension we have for every $C^2$-function $\gamma:\Bbb R\to \Bbb R$ and every $t_0,t\in\Bbb R$ $$\gamma(t)=\gamma(t_0)+\gamma'(t_0)(t-t_0)+R_1(t)$$ for some remainder $R_1(t)$ with $\lim_{t\to t_0}\frac{R_1(t)}{t-t_0}=0$. The remainder can be expressed in the Lagrange form which gives $$\gamma(t)=\gamma(t_0)+\gamma'(t_0)(t-t_0)+\frac12\gamma''(\xi)(t-t_0)^2$$ for some $\xi$ between $t$ and $t_0$. Choosing $t_0=0$, $t=1$ and $\gamma(t):=f(x+tp)$ we thus have $$\gamma(1)=\gamma(0)+\gamma'(0)+\frac12\gamma''(\xi)$$ for some $\xi\in]0,1[$. But this is exactly $$f(x+p)=f(x)+\nabla f(x)^Tp+\frac12 p^T\nabla^2f(x+\xi p)p$$ (since $$\frac{\gamma(t+s)-\gamma(t)}{s}=\frac{f(x+tp+sp)-f(x+tp)}{s}\to f'(x+tp)(p)=\nabla f(x+tp)^Tp$$ for $s\to 0$ implies that $\gamma'(t)=f(x+tp)^Tp$ and especially $\gamma'(0)=f(x)^Tp$ and similar for the second derivative).


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .