Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
we know that the Laplace Equation can be written in the form: $$\nabla^2 \Phi=0$$ while in this equation,the symbol $\nabla^2 \Phi$ stand for $\sum_{i=1}^n\frac{\partial^2 \Phi}{\partial x_i^2}$.
At the same time, the Hessian Matrix is also denoted as $$H=\nabla^2f$$
My question is what does $\nabla^2$ really mean?
Both of them are second order differential operators.
Laplacian: first outer product then inner product
Hessian : first outer product then outer product again
Usually use $\Delta$ for Laplacian to avoid confusion.
Example: outer product of outer product in 2D:
$${\bf H} = \left(\begin{array}{c} \frac{\partial}{\partial x}\\\frac{\partial}{\partial y}\end{array}\right)\left(\begin{array}{cc} \frac{\partial}{\partial x}&\frac{\partial}{\partial y}\end{array}\right) = \left(\begin{array}{cc} \frac{\partial^2}{\partial x^2} & \frac{\partial^2}{\partial xy} \\\frac{\partial^2}{\partial yx}& \frac{\partial^2}{\partial y^2}\end{array}\right)$$
$${\bf L} = \left(\begin{array}{cc} \frac{\partial}{\partial x}&\frac{\partial}{\partial y}\end{array}\right)\left(\begin{array}{c} \frac{\partial}{\partial x}\\\frac{\partial}{\partial y}\end{array}\right) = \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2}$$
In matrix language you see that Laplacian is the trace of the Hessian ${\bf L} = \rm tr({\bf H})$, i.e. that $\bf L$ is equal to the sum of the diagonal elements of $\bf H$. This is a "contraction". Reducing the number of indices of the operator from 2 to 0.
The problem with writing $\nabla^2$ is that we do not know if each application is supposed to be an inner or an outer product. For vectors there are many types of products so it is important to make sure we know which kind of product we are talking about. Therefore one should try and not use $\nabla^2$ when there can be confusion.
As pointed out by the other answer, the notation $\nabla^2$ is somewhat ambiguous (though it most commonly refers to the Laplacian). The reason is that $\nabla^2$ could be interpreted either as \begin{align} \nabla^2 &= \nabla \cdot \nabla \\ &= \left( \sum_i \mathbf{e}_i \nabla_i \right) \cdot \left( \sum_i \mathbf{e}_i \nabla_i \right) \\ &= \sum_i \nabla_i \nabla_i \\ \end{align} where $\cdot$ is the inner product, which is the Laplacian (scalar), or as \begin{align} \nabla^2 &= \nabla \otimes \nabla \\ &= \left( \sum_i \mathbf{e}_i \nabla_i \right) \otimes \left( \sum_i \mathbf{e}_i \nabla_i \right) \\ &= \sum_i \sum_j \mathbf{e}_i \mathbf{e}_j \nabla_i \nabla_j \end{align} where $\otimes$ is the outer product, which is the Hessian (matrix).
No. It doesn't have the same meaning. There is an ambiguity over the notation $\nabla^2$. In Vector Calculus, $\nabla^2$ can mean the Laplacian, which is exactly what you wrote, but also means the so-called vector Laplacian. We can differ them by noticing whether $\nabla^2$ is being applied to a potential (scalar-valued) function or a field (vector-valued) function.
In Matrix Calculus, the very same notation is also used to denote differentiation that yields the Hessian matrix. However, this computation has nothing to do with the Laplacian operator from Vector Calculus. The choice of its usage is author-biased. However, by using authoritative authors from the area of machine learning and signal processing, I highly suggest avoiding this notation for the Hessian matrix. By adopting the denominator layout, you can simply denote the Hessian matrix without any ambiguity as follows (vide Simon Haykin, Learning Machine and Neural Networks):
$$ \mathbf{H} = \frac{\partial^2 f(\mathbf{x})}{\partial \mathbf{x}^2}. $$
Differently from $\nabla^2$, this isn't a convetion. Rather, it is the differentiation operation (in denominator layout) that leads to the Hessian matrix. In numerator layout, this would be
$$ \mathbf{H} = \frac{\partial^2 f(\mathbf{x})}{\partial \mathbf{x} \partial \mathbf{x}^\top}, $$ where $\top$ denotes the transpose operation. Other notations using the outer product is still valid, but it is not so used.
Some other shorter notation such as $\nabla(\nabla f)$, $\nabla \nabla^T f$, or $\nabla \otimes \nabla f$ might be used to solve your problem, but none of them are standard and therefore I prefer the Matrix Calculus notation, IMHO.
