15
$\begingroup$

we know that the Laplace Equation can be written in the form: $$\nabla^2 \Phi=0$$ while in this equation,the symbol $\nabla^2 \Phi$ stand for $\sum_{i=1}^n\frac{\partial^2 \Phi}{\partial x_i^2}$.

At the same time, the Hessian Matrix is also denoted as $$H=\nabla^2f$$

My question is what does $\nabla^2$ really mean?

$\endgroup$

3 Answers 3

23
$\begingroup$

Both of them are second order differential operators.

Laplacian: first outer product then inner product

Hessian : first outer product then outer product again

Usually use $\Delta$ for Laplacian to avoid confusion.

Example: outer product of outer product in 2D:

$${\bf H} = \left(\begin{array}{c} \frac{\partial}{\partial x}\\\frac{\partial}{\partial y}\end{array}\right)\left(\begin{array}{cc} \frac{\partial}{\partial x}&\frac{\partial}{\partial y}\end{array}\right) = \left(\begin{array}{cc} \frac{\partial^2}{\partial x^2} & \frac{\partial^2}{\partial xy} \\\frac{\partial^2}{\partial yx}& \frac{\partial^2}{\partial y^2}\end{array}\right)$$

$${\bf L} = \left(\begin{array}{cc} \frac{\partial}{\partial x}&\frac{\partial}{\partial y}\end{array}\right)\left(\begin{array}{c} \frac{\partial}{\partial x}\\\frac{\partial}{\partial y}\end{array}\right) = \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2}$$

In matrix language you see that Laplacian is the trace of the Hessian ${\bf L} = \rm tr({\bf H})$, i.e. that $\bf L$ is equal to the sum of the diagonal elements of $\bf H$. This is a "contraction". Reducing the number of indices of the operator from 2 to 0.

The problem with writing $\nabla^2$ is that we do not know if each application is supposed to be an inner or an outer product. For vectors there are many types of products so it is important to make sure we know which kind of product we are talking about. Therefore one should try and not use $\nabla^2$ when there can be confusion.

$\endgroup$
3
  • $\begingroup$ Would inner or outer products be connected matrix language, like gradient meaning column derivative vector? $\endgroup$
    – MathArt
    Feb 12, 2021 at 9:18
  • $\begingroup$ @MathArt a gradient is outer product with $\nabla$. Gradient on a scalar becomes a vector. Gradient on a vector becomes a 2-tensor or matrix. A Jacobian could be an example. So the number of indexes increase by one. Divergence in vector analysis is an inner product with $\nabla$. Divergence of a vector becomes a scalar. Number of indices reduce by one. $\endgroup$ Feb 12, 2021 at 9:25
  • 1
    $\begingroup$ What a nice answer. $\endgroup$ Jan 4, 2023 at 18:37
13
$\begingroup$

As pointed out by the other answer, the notation $\nabla^2$ is somewhat ambiguous (though it most commonly refers to the Laplacian). The reason is that $\nabla^2$ could be interpreted either as \begin{align} \nabla^2 &= \nabla \cdot \nabla \\ &= \left( \sum_i \mathbf{e}_i \nabla_i \right) \cdot \left( \sum_i \mathbf{e}_i \nabla_i \right) \\ &= \sum_i \nabla_i \nabla_i \\ \end{align} where $\cdot$ is the inner product, which is the Laplacian (scalar), or as \begin{align} \nabla^2 &= \nabla \otimes \nabla \\ &= \left( \sum_i \mathbf{e}_i \nabla_i \right) \otimes \left( \sum_i \mathbf{e}_i \nabla_i \right) \\ &= \sum_i \sum_j \mathbf{e}_i \mathbf{e}_j \nabla_i \nabla_j \end{align} where $\otimes$ is the outer product, which is the Hessian (matrix).

$\endgroup$
1
  • 1
    $\begingroup$ Thank you for a great answer. This tradition is seriously messed up, though. $\endgroup$ Dec 6, 2021 at 17:26
0
$\begingroup$

No. It doesn't have the same meaning. There is an ambiguity over the notation $\nabla^2$. In Vector Calculus, $\nabla^2$ can mean the Laplacian, which is exactly what you wrote, but also means the so-called vector Laplacian. We can differ them by noticing whether $\nabla^2$ is being applied to a potential (scalar-valued) function or a field (vector-valued) function.

In Matrix Calculus, the very same notation is also used to denote differentiation that yields the Hessian matrix. However, this computation has nothing to do with the Laplacian operator from Vector Calculus. The choice of its usage is author-biased. However, by using authoritative authors from the area of machine learning and signal processing, I highly suggest avoiding this notation for the Hessian matrix. By adopting the denominator layout, you can simply denote the Hessian matrix without any ambiguity as follows (vide Simon Haykin, Learning Machine and Neural Networks):

$$ \mathbf{H} = \frac{\partial^2 f(\mathbf{x})}{\partial \mathbf{x}^2}. $$

Differently from $\nabla^2$, this isn't a convetion. Rather, it is the differentiation operation (in denominator layout) that leads to the Hessian matrix. In numerator layout, this would be

$$ \mathbf{H} = \frac{\partial^2 f(\mathbf{x})}{\partial \mathbf{x} \partial \mathbf{x}^\top}, $$ where $\top$ denotes the transpose operation. Other notations using the outer product is still valid, but it is not so used.

Some other shorter notation such as $\nabla(\nabla f)$, $\nabla \nabla^T f$, or $\nabla \otimes \nabla f$ might be used to solve your problem, but none of them are standard and therefore I prefer the Matrix Calculus notation, IMHO.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .