# In what case Hessian matrix off-diagnal values are zero

In the case of off-diagnal values of Hessian matrix are zero, what will be the implication?

I found a related question here.

If a function $$f$$ has the from $$f(x_1,x_2,x_3)=g_1(x_1)+g_2(x_2)+g_3(x_3)$$ then the second order derivative $$\frac{\partial^2}{\partial x_i \partial x_j} f(x) = 0 \qquad \forall i\neq j$$.

In the case of Hessian matrix, does it only implies the function is linear as @ErdelvonMises mentioned?

• When is a linear function Commented Nov 8, 2021 at 14:16
• @ErdelvonMises Could you share some reference or proof? Commented Nov 8, 2021 at 15:05
• is trivial proof it but it is no the only case Commented Nov 8, 2021 at 15:27

The other post doesn't actually solve the question as stated---let's do that. If you like you can appeal to the generalized Taylor expansion of $$f$$ to solve the problem immediately, or we can basically do the same thing with our "bare hands" below.

Suppose that $$f$$ is smooth. In showing that there are smooth functions $$g_i$$ on $$\mathbb{R}$$ such that $$f(x_1, \ldots, x_n) = g_1(x_1) + \cdots + g_n(x_n)$$, it will be slightly annoying that this decomposition is not unique if $$n > 1$$: e.g. we can always simultaneously replace $$g_1$$ with $$g_1 - 1$$ and $$g_2$$ with $$g_2 + 1$$ to get another distinct decomposition.

Instead let's impose the additional constraint that $$g_i(0) = 0$$ for all $$i$$ and that there exists a constant $$c \in \mathbb{R}$$ such that $$f(x_1, \ldots, x_n) = c + g_1(x_1) + \cdots + g_n(x_n)$$. We will show that such a decomposition exists and is unique when the Hessian of $$f$$ is diagonal.

Let's first guess the functions $$g_i$$: we better have that $$g_i'(t) = \frac{d f}{d x_i}(0, \ldots, 0) + \int_0^t \frac{d^2 f}{d x_i^2}(0, \ldots, 0, t, 0, \ldots, 0) d t,$$ i.e. so that $$g_i(t) := \frac{d f}{d x_i}(0, \ldots, 0) t + \int_0^t \int_0^s \frac{d^2 f}{d x_i^2}(0, \ldots, 0, s, 0, \ldots, 0) d s d t.$$ (Recall that we want $$g_i(0) = 0$$.) Then we also need $$c := f(0, \ldots, 0)$$.

Ok, so now put $$h(x_1, \ldots, x_n) = f(0, \ldots, 0) + g_1(x_1) + \cdots + g_n(x_n)$$: we claim $$f = h$$. Indeed, for any $$(x_1, \ldots, x_n) \in \mathbb{R}^n$$ by the ordinary 1-dimensional fundamental theorem of calculus we have $$f(x_1, 0, \ldots, 0) - f(0, \ldots, 0) = \int_0^{x_1} \frac{d f}{d x_1}(t, 0, \ldots, 0) d t,$$ so that inductively we have \begin{align*} f(x_1, \ldots, x_n) - f(0, \ldots, 0) &= \int_0^{x_1} \frac{d f}{d x_1}(t, 0, \ldots, 0) d t + \int_0^{x_2} \frac{d f}{d x_2}(x_1, t, \ldots, 0) d t\\ &\phantom{===}+ \cdots + \int_0^{x_n} \frac{d f}{d x_n}(x_1, \ldots, x_{n - 1}, t) d t. \end{align*} If we can show that $$\frac{d f}{d x_k}(x_1, \ldots, x_{k - 1}, t, x_{k + 1}, \dots, x_n) = \frac{d f}{d x_k}(0, \ldots, 0, t, 0, \dots, 0)$$ for all $$k$$ then we will be done, since then the previous equation just becomes \begin{align*} f(x_1, \ldots, x_n) - f(0, \ldots, 0) &= \int_0^{x_1} g_1'(t) d t + \int_0^{x_2} g_2'(t) d t + \cdots + \int_0^{x_n} g_n'(t) d t\\ &= g_1(x_1) + \cdots + g_n(x_n), \end{align*} as desired.

To check that $$\frac{d f}{d x_k}(x_1, \ldots, x_{k - 1}, t, x_{k + 1}, \dots, x_n) = \frac{d f}{d x_k}(0, \ldots, 0, t, 0, \dots, 0)$$ we finally use the hypothesis that off-diagonal entries of the Hessian of $$f$$ vanish: note just like before by the fundamental theorem of calculus we have \begin{align*} \frac{d f}{d x_k}(x_1, \ldots, 0, t, 0, \dots, 0) - \frac{d f}{d x_k}(0, \ldots, 0, t, 0, \dots, 0) &= \int_0^{x_1} \frac{d}{d x_1} \frac{d f}{d x_k}(s, \ldots, 0, t, 0, \dots, 0) d s\\ &= 0, \end{align*} since $$\frac{d}{d x_1} \frac{d f}{d x_k} = 0$$ (because $$k \not = 1$$). so that inductively we can conclude \begin{align*} \frac{d f}{d x_k}(x_1, \ldots, x_{k - 1}, t, x_{k + 1}, \dots, x_n) - \frac{d f}{d x_k}(0, \ldots, 0, t, 0, \dots, 0), \end{align*} as desired.

First recall the definition of the Hessian $$H$$ of a function $$f$$ $$H_{i,j} = \partial_{x_i,x_j} f$$ For all the off-diagonal term to be zero (e.i. $$H_{i,j} = 0 \;\;\;\; \forall ij : i \neq j$$) $$0 = \partial_{x_i,x_j} f \;\;\;\; \forall ij : i \neq j$$ must happen, suppose $$f$$ is a linear combination of functions with dependency in no more than a variable $$f = \sum_{m=1}^M a_m g_m(x_m)$$ so the $$0 = \partial_{x_i,x_j} f \;\;\;\; \forall ij : i \neq j$$ of $$f$$ is $$\partial_{x_i,x_j} f = \sum_{m=1}^M a_m \partial_{x_i,x_j} g_m(x_m) = 0$$ This trivially implies that if $$f$$ is a linear combination of functions with dependency in no more than a variable, then the off-diagonal entries will be zero. And I conjecture the before mentioned condition is nesesary and sufficient to the off-diagonal entriess of the Hessian be zero.

For gain some intuition of why it would no apply to other cases. Suppose $$f$$ is a multivariate polynomial of degree two $$f = a + \sum_{n = 1}^N \sum_{m=1}^2 x_n^m + \sum_{n = 1}^N \sum_{m = 1}^M c_{n,m} x_n x_m$$ We again we get the Hessian off-diagonal derivatives $$\partial_{x_i,x_j} f = \sum_{n = 1}^N \sum_{m = 1}^M c_{n,m} \partial_{x_i,x_j} x_n x_m \neq 0 \,\,\,\, \text{for} \,\, i \neq j$$