We are about to look at an important type of matrix in multivariable calculus known as Hessian Matrices. We will then formulate a generalized second derivatives test for a real-valued function $z=f(x_1,x_2,...,x_n)$ of n variables with continuous partial derivatives at a critical point $a=(a1,a2,...,an)∈D(f)$ to determine whether $f(a)$ is a local maximum value, local minimum value, or saddle point of $f$
Definition: Let $x=(x1,x2,...,xn)$ and let $z=f(x1,x2,...,xn)=f(x)$ be an n variable real-valued function whose second partial derivatives exist. Then the Hessian Matrix of f is the n×n matrix of second partial derivatives of f denoted $$\mathcal H (\mathbf{x}) = \begin{bmatrix} f_{11} (\mathbf{x}) & f_{12} (\mathbf{x}) & \cdots & f_{1n} (\mathbf{x})\\ f_{21} (\mathbf{x}) & f_{22} (\mathbf{x}) & \cdots & f_{2n} (\mathbf{x})\\ \vdots & \vdots & \ddots & \vdots \\ f_{n1} (\mathbf{x}) & f_{n2} (\mathbf{x}) & \cdots & f_{nn} (\mathbf{x}) \end{bmatrix}$$.
a)$\mathcal H (\mathbf{x})$ is said to be $\textbf{Positive Definite}$ if $D_i>0$ for i=1,2,...,n.
b) $\mathcal H (\mathbf{x})$ is said to be $\textbf{Negative Definite}$ if $D_i<0$ for odd i∈{1,2,...,n} and Di>0 for even i∈{1,2,...,n}.
c) $\mathcal H (\mathbf{x})$ is said to be $\textbf{Indefinite}$ if $det(\mathcal H (\mathbf{x}))=D_n≠0$ and neither a) nor b) hold.
d) If $det(\mathcal H (\mathbf{x}))=D_n=0$, then $\mathcal H (\mathbf{x})$ may be Indefinite or what is known Positive Semidefinite or Negative Semidefinite.
I'm studying a function that has a $det(\mathcal H (\mathbf{x}))=D_n=0$
What is the appropriate way to classify critical points if $det(\mathcal H (\mathbf{x}))=D_n=0$
For example Assume we have this function $$f(x_1,x_2,x_3)=x_1 x_2+x_1 x_3-x_1x_2 x_3 $$ therefore, we found the critical point by solve this system $$\begin{equation} \begin{cases} \frac{df}{dx_1}=x_2+x_3-x_3 x_2=0\\ \frac{df}{dx_2}=x_1-x_1 x_3 =0\\ \frac{df}{dx_3}=x_1-x_1 x_2 =0 \end{cases} \end{equation}$$ we get $c=(0,0,0)$ ,use the Hessian matrix of f $$\mathcal H (\mathbf{f(x_i)})=\left( \begin{array}{ccc} 0 & 1-x_3 & 1-x_2 \\ 1-x_3 & 0 & -x_1 \\ 1-x_2 & -x_1 & 0 \\ \end{array} \right)$$ $det(\mathcal H (\mathbf{f(x_i)}))=D_n=0$
This test fails to determine the type of critical points
Is there another way? Or modification of the solution and correction. Thanks for the help.