# At a Critical Point the Hessian Equals the Frst Nonconstant Term in the Taylor Series of $f$?

My textbook defines Hessian functions as follows:

Suppose that $$f: U \subset \mathbb{R}^n \to \mathbb{R}$$ has second-order continuous derivatives $$\dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0)$$, for $$i, j = 1, \dots, n$$, at a points $$\mathbf{x}_0 \in U$$. The Hessian of $$f$$ at $$\mathbf{x}_0$$ is the quadratic function defined by

\begin{align} Hf(\mathbf{x}_0)(\mathbf{h}) &= \dfrac{1}{2} \sum_{i, j = 1}^n \dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0) h_i h_j \\ &= \dfrac{1}{2} [h_1, \dots, h_n] \left[\begin{matrix}\frac{\partial^2 f}{\partial x_1^2} & \ldots & \frac{\partial^2 f}{\partial x_1\partial x_n}\\ \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n\partial x_1}& \ldots & \frac{\partial^2 f}{\partial x_n^2}\end{matrix}\right] \left[\begin{matrix} h_1 \\ \vdots \\ h_n \end{matrix}\right] \end{align}

Notice that, by equality of mixed partials, the second-derivative matrix is symmetric.

This function is usually used at critical points $$\mathbf{x}_0 \in U$$. In this case, $$\mathbf{D}f(\mathbf{x}_0) = \mathbf{0}$$, so the Taylor formula (see Theorem 2, section 3.2) may be written in the form

$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + Hf(\mathbf{x}_0)(\mathbf{h}) + R_2(\mathbf{x}_0, \mathbf{h})$$

Thus, at a critical point the Hessian equals the first nonconstant term in the Taylor series of $$f$$.

Theorem 2, section 3.2, is as follows:

Theorem 2 First-Order Taylor Formula

Let $$f: U \subset \mathbb{R}^n \to \mathbb{R}$$ be differentiable at $$\mathbf{x}_0 \in U$$. Then

$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + \sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0) + R_1(\mathbf{x}_0, \mathbf{h}),$$

where $$\dfrac{R_1(\mathbf{x}_0, \mathbf{h})}{|| \mathbf{h} ||} \to 0$$ as $$\mathbf{h} \to \mathbf{0}$$ in $$\mathbb{R}^n$$.

My question is, what is meant by this:

Thus, at a critical point the Hessian equals the first nonconstant term in the Taylor series of $$f$$.

I would greatly appreciate it if people could please take the time to clarify this.

EDIT:

I think the authors actually meant to refer to theorem 3, section 3.2:

Theorem 3 Second-Order Taylor Formula

Let $$f: U \subset \mathbb{R}^n \to \mathbb{R}$$ have continuous partial derivatives of third order. Then we may write

$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + \sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0) + \dfrac{1}{2} \sum_{i, j = 1}^n h_i h_j \dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0) + R_2(\mathbf{x}_0, \mathbf{h}),$$

where $$\dfrac{R_2(\mathbf{x}_0, \mathbf{h})}{|| \mathbf{h} ||^2} \to 0$$ as $$\mathbf{h} \to \mathbf{0}$$ and the second sum is over all $$i$$'s and $$j$$'s between $$1$$ and $$n$$ (so there are $$n^2$$ terms).

• Theorem 3 looks like a second-order Taylor formula to me. – Joppy Oct 24 '18 at 2:21
• @Joppy You're right, I made a typo. Thanks for that. – The Pointer Oct 24 '18 at 5:20

Well the first term in the Taylor series is $$f(x_0)$$. This doesn't depend on $$h$$ and so we are calling it `constant'. Usually the next term contains $$\nabla f(x_0)$$, but at a critical point, this is zero. Therefore the first term in the Taylor series that depends on $$h$$ is the term involving the Hessian.
• Thanks for the clarification. In the above theorems, I think $\sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0)$ is the same as $\nabla f(x_0)$. – The Pointer Oct 24 '18 at 2:19
• Or, rather, I should say that $\sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0)$ is the same as $[\nabla f(x_0)](\mathbf{h})$ – The Pointer Oct 24 '18 at 5:22