Determining positive/negative definite of quadratic form using Hessian matrix method? The matrix A is given as:
\begin{bmatrix}0&1&1\\1&0&1\\1&1&0\end{bmatrix}
Given that the domain of the quadratic form $x^T Ax$ is restricted as the following,
$D={x\in R^3, x_1+x_2+x_3=0}$
determine whether the quadratic form is positive/negative definite or positive/negative semidefinite.
I know how to solve this kind of problem when there's no restriction/constraint, but I have no idea how to determine this when there is a restriction. The textbook talks about some bordered Hessian matrix method and some other Hessian matrix method but I have no idea how to apply these methods...
 A: $A=\begin{bmatrix}0&1&1\\1&0&1\\1&1&0\end{bmatrix} = \mathbf {11}^T-I$
and you want to know the signature with the added restriction that we only consider $\mathbf x$ such that $\mathbf 1^T\mathbf x = 0$.
There are a few different approaches here.  Probably the simplest is to make use of the projection matrix
$P:= I-\alpha\mathbf {11}^T$, where $\alpha$ is selected so $P$ has rank $n-1$, i.e. $\alpha =\Big(\mathbf 1^T \mathbf 1\Big)^{-1} = n^{-1}$
check:  $P^2=P=P^T$
$B:= P^TAP$  and notice that if $\mathbf x \perp \mathbf 1$
$P\mathbf x = \mathbf x \implies\mathbf x^T B \mathbf x = \mathbf x^T A \mathbf x$
$B$ is real symmetric with $\mathbf 1$ in its kernel so you may in general orthogonally diagonalize it and consider the $n-1$ orthogonal eigenvvectors of interest to figure out its signature (or run $LDL^T$ factorization, etc.)
That said, your problem is particularly simple:
$B= P^TAP =  P^T\big(\mathbf {11}^T-I\big)P = -P^TP = -P $
since $-P$ has signature $\big(0,n-1\big)$ with only vectors $\propto\mathbf 1$ in its kernel this tells you that $\mathbf x^T A\mathbf x \lt 0$ for any non-zero $\mathbf x \perp \mathbf 1$
A: We have that
$$x^T Ax =2x_1x_2+2x_2x_3+2x_3x_1$$
and for any $x=(x_1,x_2,x_3)\neq 0$
$$(x_1+x_2+x_3)^2=0 \implies  2x_1x_2+2x_2x_3+2x_3x_1 =-(x_1^2+x_2^2+x_3^2)<0$$
therefore by this restriction the quadratic form is negative definite.
