eigenvalues of bordered Hessian $$
\newcommand{\mb}[1]{\mathbf{#1}}
\newcommand{\bs}[1]{\boldsymbol{#1}}
\newcommand{\mleft}{\left}
\newcommand{\mright}{\right}
$$
I'm interested in a general statement on the eigenvalues of the bordered Hessian. The bordered Hessian is arising from optimization with equality constraints in a Lagrange-multiplier framework.
We optimize a function $f(\mb{x})$ over an $n$-dimensional vector $\mb{x}$. There are $m$ equality constraints $g_i(\mb{x}) = 0$, summarized in a vector $\mb{g}(\mb{x})$. The Lagrangian (with Lagrange multipliers $\bs{\lambda}$) is given by
$$
\begin{align}
L\mleft( \mb{x},\bs{\lambda} \mright)
&=
f\mleft( \mb{x} \mright) + \bs{\lambda}^T \mb{g}\mleft( \mb{x} \mright)
\end{align},
$$
the first-order derivatives are
$$
\begin{align}
\frac{\partial}{\partial \mb{x}}L\mleft( \mb{x},\bs{\lambda} \mright)
&=
\frac{\partial}{\partial \mb{x}}f\mleft( \mb{x} \mright) +
\bs{\lambda}^T \mleft( \frac{\partial}{\partial \mb{x}}\mb{g}\mleft(
\mb{x} \mright) \mright)\\
%====================
\frac{\partial}{\partial \bs{\lambda}}L\mleft( \mb{x},\bs{\lambda}
\mright)
&=
\mleft( \mb{g}\mleft( \mb{x} \mright) \mright)^T
\end{align}
$$
and the bordered Hessian contains the second order derivatives
$$
\begin{align}
\bar{\mb{H}}
&=
\begin{pmatrix}\frac{\partial}{\partial \mb{x}}\mleft(
\frac{\partial}{\partial \mb{x}}L\mleft( \mb{x},\bs{\lambda} \mright)
\mright)^T & \frac{\partial}{\partial \bs{\lambda}}\mleft(
\frac{\partial}{\partial \mb{x}}L\mleft( \mb{x},\bs{\lambda} \mright)
\mright)^T\\\frac{\partial}{\partial \mb{x}}\mleft(
\frac{\partial}{\partial \bs{\lambda}}L\mleft( \mb{x},\bs{\lambda}
\mright) \mright)^T & \mb{0}_{m,m}\end{pmatrix}
\end{align}
$$
with
$$
\begin{align}
\frac{\partial}{\partial \mb{x}}\mleft( \frac{\partial}{\partial
\mb{x}}L\mleft( \mb{x},\bs{\lambda} \mright) \mright)^T
&=
\frac{\partial}{\partial \mb{x}}\mleft( \frac{\partial}{\partial
\mb{x}}f\mleft( \mb{x} \mright) \mright)^T + \mleft( \sum\limits_{k =
1}^{m}\lambda_{k} \mleft[ \frac{\partial}{\partial x_{j}}\mleft\{
\frac{\partial}{\partial x_{i}}g_{k}\mleft( \mb{x} \mright) \mright\}
\mright] \mright)^{n\times n}_{i,j}
\end{align}
$$
and
$$
\begin{align}
\frac{\partial}{\partial \bs{\lambda}}\mleft( \frac{\partial}{\partial
\mb{x}}L\mleft( \mb{x},\bs{\lambda} \mright) \mright)^T
&=
\mleft( \frac{\partial}{\partial \mb{x}}\mb{g}\mleft( \mb{x} \mright)
\mright)^T
\end{align}
$$
(note that any Hessian is symmetric).
I'm interested in the question whether $L(\mb{x},\bs{\lambda})$ has a saddle point at each fixed point of the first-order derivatives when the vector space is defined by $\mb{x}$ together with $\bs{\lambda}$.
G.R. Walsh (Methods of Optimization, Wiley, 1975) gives a Theorem (1.1, p.20) which states that it is a saddle point. I'm afraid I don't understand the proof; it seems to be based on the observation that a quadratic form of the bordered Hessian is zero for vectors where the components corresponding to $\mb{x}$ are zero.
D. Kalman (Leveling with Lagrange: An Alternate View of Constrained Optimization, Mathematics Magazine, 82:3, 186-196, 2009) provides a proof of the saddle-point property for the case of $n=2$ and $m=1$, but I'm not sure how this generalizes to arbitrary $n$ and $m$.
J. Baker (Geometry Optimization in Cartesian Coordinates: Constrained Optimization, J. Comp. Chem., 13:2, 240-253, 1992) writes "what typically happens ... is that the inclusion of the constraint introduces an additional mode ... whose largest component corresponds to the ... Lagrange multiplier ... and which has ... a negative Hessian eigenvalue ..." and "... each constraint results in an additional mode with negative curvature". There are also statements that the original modes are modified to some degree by the Lagrange multipliers (I assume that they retain their positive eigenvalues, but this is not clear). Unfortunately, no citation or analysis is provided.
So there seems to be some knowledge on the eigenvalues of the bordered Hessian, but looking at the equation of the bordered Hessian above I have no idea how they can be derived. I would be really grateful for some ideas or pointers to the literature. Thanks a lot for your help!
 A: the Eigen values of a Matrix M (for you the Hessian Hnxn) are the solutions of the equation: determinant (M – λI)  =  0, where λ is the the unknown Eigen value and I is the nxn identity matrix. Refer to this link for more info: https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors#Eigenvalues_and_the_characteristic_polynomial                                  Also, you can use the Wolfram Alpha online which will compute the Eigen values from your matrix H input: https://www.wolframalpha.com/input/?i2d=true&i=eingen+values
A: Below are the "rules" for local optima of a function using the Hessian and associated Eigenvalues:
(I).    Local Minima:

*

*A twice-differentiable real function f(x) on n real variables, x1,, x2,…, x n,  has local minimum at arguments x* =  x1,, x2,…, x n , if its gradient is zero and its Hessian H (the matrix of all second derivatives) is positive semi-definite at x*

*Similarly, f(x) has a local maximum at x*, if its gradient is zero and its Hessian H  is negative semi-definite if and only if all of its eigenvalues are non-negative at x*.
(II).    Rule for “definiteness” of a real Matrix M:

*M is positive definite if and only if all of its eigenvalues are positive.

*M is positive semi-definite if and only if all of its eigenvalues are non-negative.

*M is negative definite if and only if all of its eigenvalues are negative

*M is negative semi-definite if and only if all of its eigenvalues are non-positive.

*M is indefinite if and only if it has both positive and negative eigenvalues.

*M is positive definite If and only if all  its leading principal minors are all positive. The kth leading principal minor of a matrix is the determinant of its upper-left k x k  sub-matrix.

*A matrix M is negative (semi)definite if and only if -M  is positive (semi)definite.
(III). Combining (I) and (II)


*

*If the Hessian is positive definite (equivalently (has all eigenvalues positive), at x*,  then f attains a local minimum at x*.

*If the Hessian is negative definite (equivalently,  (has all eigenvalues negative) at x*, then f attains a local maximum at x*.

*If the Hessian has both positive and negative eigenvalues then a is a saddle point for f (and in fact this is true even if a is degenerate).

References: https://www.google.com/search?q=positive+definite+matrix+properties&rlz=1C1GCEA_enES775ES775&oq=positive+definite&aqs=chrome.3.0i512j69i57j0i512l8.21212j0j7&sourceid=chrome&ie=UTF-8
https://en.wikipedia.org/wiki/Second_partial_derivative_test
