My textbook defines Hessian functions as follows:

Suppose that $f: U \subset \mathbb{R}^n \to \mathbb{R}$ has second-order continuous derivatives $\dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0)$, for $i, j = 1, \dots, n$, at a points $\mathbf{x}_0 \in U$. The Hessian of $f$ at $\mathbf{x}_0$ is the quadratic function defined by

\begin{align} Hf(\mathbf{x}_0)(\mathbf{h}) &= \dfrac{1}{2} \sum_{i, j = 1}^n \dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0) h_i h_j \\ &= \dfrac{1}{2} [h_1, \dots, h_n] \left[\begin{matrix}\frac{\partial^2 f}{\partial x_1^2} & \ldots & \frac{\partial^2 f}{\partial x_1\partial x_n}\\ \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n\partial x_1}& \ldots & \frac{\partial^2 f}{\partial x_n^2}\end{matrix}\right] \left[\begin{matrix} h_1 \\ \vdots \\ h_n \end{matrix}\right] \end{align}

Notice that, by equality of mixed partials, the second-derivative matrix is symmetric.

This function is usually used at critical points $\mathbf{x}_0 \in U$. In this case, $\mathbf{D}f(\mathbf{x}_0) = \mathbf{0}$, so the Taylor formula (see Theorem 2, section 3.2) may be written in the form

$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + Hf(\mathbf{x}_0)(\mathbf{h}) + R_2(\mathbf{x}_0, \mathbf{h})$$

Thus, at a critical point the Hessian equals the first nonconstant term in the Taylor series of $f$.

Theorem 2, section 3.2, is as follows:

Theorem 2 First-Order Taylor Formula

Let $f: U \subset \mathbb{R}^n \to \mathbb{R}$ be differentiable at $\mathbf{x}_0 \in U$. Then

$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + \sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0) + R_1(\mathbf{x}_0, \mathbf{h}),$$

where $\dfrac{R_1(\mathbf{x}_0, \mathbf{h})}{|| \mathbf{h} ||} \to 0$ as $\mathbf{h} \to \mathbf{0}$ in $\mathbb{R}^n$.

My question is, what is meant by this:

Thus, at a critical point the Hessian equals the first nonconstant term in the Taylor series of $f$.

I would greatly appreciate it if people could please take the time to clarify this.


I think the authors actually meant to refer to theorem 3, section 3.2:

Theorem 3 Second-Order Taylor Formula

Let $f: U \subset \mathbb{R}^n \to \mathbb{R}$ have continuous partial derivatives of third order. Then we may write

$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x}_0) + \sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0) + \dfrac{1}{2} \sum_{i, j = 1}^n h_i h_j \dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\mathbf{x}_0) + R_2(\mathbf{x}_0, \mathbf{h}),$$

where $\dfrac{R_2(\mathbf{x}_0, \mathbf{h})}{|| \mathbf{h} ||^2} \to 0$ as $\mathbf{h} \to \mathbf{0}$ and the second sum is over all $i$'s and $j$'s between $1$ and $n$ (so there are $n^2$ terms).

  • 1
    $\begingroup$ Theorem 3 looks like a second-order Taylor formula to me. $\endgroup$ – Joppy Oct 24 '18 at 2:21
  • $\begingroup$ @Joppy You're right, I made a typo. Thanks for that. $\endgroup$ – The Pointer Oct 24 '18 at 5:20

Well the first term in the Taylor series is $f(x_0)$. This doesn't depend on $h$ and so we are calling it `constant'. Usually the next term contains $\nabla f(x_0)$, but at a critical point, this is zero. Therefore the first term in the Taylor series that depends on $h$ is the term involving the Hessian.

  • $\begingroup$ Thanks for the clarification. In the above theorems, I think $\sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0)$ is the same as $\nabla f(x_0)$. $\endgroup$ – The Pointer Oct 24 '18 at 2:19
  • $\begingroup$ Or, rather, I should say that $\sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0)$ is the same as $[\nabla f(x_0)](\mathbf{h})$ $\endgroup$ – The Pointer Oct 24 '18 at 5:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.