Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hessian-matrix]

The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both ...

3
votes
1answer
48 views

Hessian of a $C^{\infty}$ function $f: \mathbb R^n \to \mathbb R$ restricted to a subspace

Let $f: \mathbb R^n \to \mathbb R$ be an infinitely differentiable function. Let $\mathcal V \subset \mathbb R^n$ be a vector subspace. Let $g = f|_{\mathcal V}$ be the restriction of $f$. Then we can ...
0
votes
0answers
29 views

Hessian decomposition?

I came across the decomposition below but I am not sure how to derive it or if it is in fact true. Can anyone share some insight? $D^2v=\partial^2_{nn}v(n^T)+\partial^2_{nr}v (nr^T+rn^T)+\partial^2_{...
1
vote
0answers
21 views

How to classify the degenerate critical (stationary) points of a multivariate function?

I have a multivariate function in one of whose critical points the Hessian matrix is singular. Is there any general method to determine the type of this critical point? Would be worth plotting the ...
0
votes
1answer
23 views

what if the hessian of the Lagrangian is null?

In a non-linear optimization problem, suppose given a candidate solution $x*$, we want to verify second order sufficient conditions of optimality but the Hessian of the Lagrangian of the problem at $x*...
4
votes
0answers
31 views

A matrix problem about egienvalue and trace

Consider an $m\times m$ positive definite and Hermitian matrix $\mathbf{M}$ and an arbitrary $m\times n (m>n)$ para-unitary matrix $\mathbf{R}$, i.e., $\mathbf{R}^H\mathbf{R}=\mathbf{I}_n$. ...
0
votes
2answers
27 views

low-complexity matrix inversion algorithm for a near-identity matrix?

As I know, the general complexity of matrix inversion is $O(n^3)$, but it is a little bit high. My matrix is $(I + A)$ , where $I$ is an $n \times n$ identity matrix and $A$ is a hermitian matrix ...
4
votes
3answers
56 views

Gradient and Hessian of $x x^T$ w.r.t. $x$, where $x \in \mathbb{R}^{n \times 1}$,?

Question: Can we find the gradient and Hessian of $x x^T$ w.r.t. $x$, where $x \in \mathbb{R}^{n \times 1}$ ? EDIT: If we can, may I know how to compute that? Thank you.
1
vote
0answers
27 views

Condition number of Hessian matrix when Hessian is singular

In gradient descent on a quadratic problem, $$\min _{x \in \mathbb{R}^n}\frac12 \langle Hx, x \rangle + \langle b, x \rangle + c \qquad H \text{ symmetric, positive semi-definite}\quad b \in \...
3
votes
2answers
70 views

Calculate the Hessian of a Vector Function

I'm working with optimisation. I am trying to obtain the hessian of a vector function: $$ \mathbf{F(X) = 0} \quad \text{or} \quad \begin{cases} f_1(x_1,x_2,\dotsc,x_n) = 0,\\ f_2(x_1,x_2,\...
0
votes
2answers
55 views

Example of convex function which is differentiable, but not twice differentiable?

Are there convex functions for which hessian is not defined, but the gradient is defined everywhere? I was looking at projected gradient descent, as well as Newton's method for solving optimization ...
0
votes
1answer
17 views

Second directional derivative and Hessian matrix: a clarification on the proof

I have been reading this answer and I couldn't get the proof. The argument is about how we can write the second directional derivative of $f:\mathbf{R}^m\rightarrow\mathbf{R}$ with respect to ...
3
votes
1answer
28 views

How to check for local extrema or saddle point given an semidefinite matrix

I've computed the Hessian of a given function $f(a,b,c) = y-a\sin(bx-c)$ and got the following result: $\begin{pmatrix} 0 & -x\cdot\cos(bx - c) & \cos(bx - c) \\ -x\cdot\cos(bx - c) & ax^...
2
votes
3answers
118 views

What is the higher-order derivative test in multivariable calculus?

In single-variable calculus, the second-derivative test states that if $x$ is a real number such that $f'(x)=0$, then: If $f''(x)>0$, then $f$ has a local minimum at $x$. If $f''(x)<0$, then $...
1
vote
0answers
44 views

Hessian matrix vs differential 2-form

Could someone clarify the convention that the second derivative of a scalar function $f: \Bbb R^n \rightarrow \Bbb R$ is sometimes defined as a linear operator $D^2f : \Bbb R^n \rightarrow L(\Bbb R^n, ...
2
votes
5answers
111 views

Hessian on linear least squares problem

I tried to calculate the Hessian matrix of linear least squares problem (L-2 norm), in particular: $$f(x) = \|AX - B \|_2$$ where $f:{\rm I\!R}^{11\times 2}\rightarrow {\rm I\!R}$ Can someone help ...
2
votes
0answers
61 views

Sign of the Bakry-Emery curvature operator $\Gamma_2(f) :=|Hess f|^2+Ric(\nabla f,\nabla f)$

I am not an expert of Riemannian geometry (coming mainly from functional analysis in $\mathbb R^n$). Let $(M,g)$ be a compact $n$-dimensional Riemannian manifold with Ricci curvature bounded from ...
0
votes
0answers
26 views

Approximating Hessian with BFGS for a matrix variable

So as we know the approximation of the inverse of the hessian matrix using the BFGS method is calculated with the following formulas : $$q_{k+1} = (I-p_k s_k (y_k)^T)q_k(I- p_k y_k (s_k)^T) + p_k ...
0
votes
0answers
33 views

Reading Help - Hessian, Linear Algebra, Finding Eigenvalues

Edit: I've cut some parts to try and focus this question on linear algebra I'm having a lot of difficulty understanding some parts of this paper, which I'm reading out of personal interest, yet am ...
1
vote
1answer
33 views

When does $f(x_0)$ maximum $\implies \det \mathrm{Hess}_f(x_0)\neq 0$?

Let $f:\mathbb{R}^n\to\mathbb{R}$ be a smooth function and let $x_0\in \mathbb{R}^{n+1}$ be a critical point of $f$ such that $f(x_0)$ is a local maximum. Question: Under which conditions does it ...
0
votes
1answer
38 views

Local minimum of an analytic function

This is a follow-up to a previous question of mine. I know that any local minimum $x_0$ of a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ has positive semi-definite Hessian $H(x_0) \succeq 0$. ...
3
votes
2answers
36 views

Local minimum has neighbourhood with positive semi-definite Hessian?

I know that any local minimum $x_0$ of a (twice continuously differentiable) function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ has positive semi-definite Hessian $H(x_0) \succeq 0$. Can we say ...
0
votes
1answer
11 views

Properties of norms of inverse Hessians near a solution

In theorem 3.5 of the book on Numerical Optimization by Jorge Nocedal and Stephen J Wright, Second edition, is given the proof for the quadratic convergence to the solution of a sequence of iterates ...
0
votes
1answer
38 views

Finding the extrema of $f(x,y)$ when ALL critical points $=0$

Let $f: \mathbb R^{2}\to\mathbb R$ and $f(x,y)=(x^2-1)(y-1)^2$. Find all the extrema and saddle points. Idea: Setting $\nabla f(x,y)=0$, we get: i) $2x(y-1)^2=0$ ii) $(x^2-1)2(y-1)=0$ Therefore ...
0
votes
0answers
33 views

Convex inequality is twice differentiable with Hessian matrix

Let $f: A\subseteq \mathbb{R}^n \rightarrow \mathbb{R}$ be continuously differentiable such that $$\nabla^T f:A\rightarrow\mathbb{R}^n, x\mapsto [\delta_1 f(x),\dots,\delta_n f(x)]^T$$ is continuously ...
0
votes
1answer
20 views

Evaluating an extremal value if the hessian matrix has at least one eigenvalue which is zero

$$f(x,y) = 2x^4-3x^2y + y^2$$ We want to find all extremal values: $$df(x,y)=(8x^3-xy,-3x^2+2y)\overset{!}{=}0 \quad \Rightarrow \quad p=(0,0)$$ $$H_f(x,y)=\begin{pmatrix}24x^2-6y& -6x\\-6x &...
-1
votes
2answers
40 views

Extrema when Hessian Matrix is $0$ at $(0,2)$

Let $f: \mathbb R^{2}\to \mathbb R$, $f(x,y)=xy^2-4xy+x^4$ Find all critical points of $f$ and determine whether or not they are extrema and their type: Setting $0=\nabla f(x,y)$ I get the ...
0
votes
0answers
59 views

Interpretation of the magnitude of an eigenvalue?

We know that the eigenvalues of a Hessian matrix can provide information about the curvature of a function under study. Specifically, we know that the sign of the eigenvalues give us information ...
0
votes
0answers
38 views

Local expression of Hessian operator in Riemannian Geometry

Define the Hessian of a differentiable function $F \colon M \to \mathbb{R}$ where $M$ is a Riemannian manifold by $$(\nabla^2 F)(X,Y) = X(Y(F)) - (\nabla_X Y)(F)$$ where $\nabla$ is the connection on ...
0
votes
1answer
42 views

How to obtain the following result?

I have a function $f(x)=h([g_1(x),~g_2(x),\ldots, g_k(x)])$ and I want to find $f''(x)$. To this end, first I have $$f'(x)=\nabla h([g_1(x),~g_2(x),\ldots, g_k(x)]) \left[ \begin{array}{c} g_1'(x) \\...
1
vote
0answers
89 views

Bifurcation, critical points and parameter dependence in $\nabla_r\,\phi_a(r) = 0$ when varying $a$

Suppose I have a function $\phi_a(r)$, where $r \in \mathbb{R}^n$ denotes a real n-dim. vector, and where $a \in \mathbb{R}^p$ denotes a set of additional real parameters. Suppose that for a given $...
0
votes
0answers
21 views

Positive semi-definiteness of the Jacobian of the partial derivatives of distinct convex functions

Let $f_i: \mathbb{R}^{Nn} \rightarrow \mathbb{R}$ be convex and twice continously differentiable functions for every $i=1...N$. Consider the function $F: \mathbb{R}^{Nn} \times \mathbb{R}^{Nn}$ ...
1
vote
1answer
49 views

Can this log function have infinitely many saddle points?

The question asks me to find and classify all the critical points of: $$f(x,y)=\text{ln}(1+x^2y)$$ My solution: $$f'_x = \frac{2xy}{1+x^2y}; f'_y=\frac{x^2}{(1+x^2y)}; f''_{xx}=\frac{(2y)(1+x^2y)-(2xy)...
0
votes
0answers
16 views

Two methods for second order sufficient conditions for constrained optimization

There are at least two methods to verify those type of conditions stated in their weaker form: projected Hessian bordered Hessian In these course notes (section 2.6 Projected Hessians - page 19 in ...
1
vote
1answer
34 views

Quadratic function optimum not aligning with implementation.

Let's define a quadratic function ($x$, $x_0$ and $g$ are vectors and $H$ is a symmetric matrix so, $H^T=H$): $$f(x) = \frac{(x-x_0)^TH(x-x_0)}{2}+g^T(x-x_0)$$ $$=>f(x)=\frac{x^THx}{2}+\frac{x_0^...
0
votes
0answers
17 views

Condition number of the indefinite Hessian

I compute Hessians on random samples of my non-convex high-dimensional optimisation problem to get a better understanding of the function shape. I thought I will quantify the amount of ill-...
0
votes
0answers
24 views

Cardboard box Optimization Problem To Maximize Size and Minimize Cost

I am requesting some help or guidance for the problem pictured above. It is similar to a problem in the textbook we are using with some added curve balls that have me stuck. Should I be using the ...
0
votes
0answers
21 views

Eigen values of the Hessian of a cost function

While performing an optimization, we try to minimize (or maximize) a cost function. What information about the optimizer can be extracted when we look at the eigen values of the Hessian of the cost ...
0
votes
0answers
25 views

CES function quasi-concave?

I have the following CES (constant elasticity of substitution) function: $y=\left(\beta \ c^s + (1-\beta ) \ d^s\right)^{\frac{1}{s}}$ with $0<\beta<1$, $-\infty\leq s \leq1$ and both $c,d$ $&...
1
vote
1answer
28 views

Hessian of the norm of a non-linear map

Suppose $F: \mathbb{R}^n \rightarrow \mathbb{R}^m$ and define the scalar valued map $\Phi(x;y) = \frac{1}{2}\|y - F(x)\|_2^2 $. I am interested in the Hessian of this map written in terms of the ...
1
vote
0answers
23 views

Chaining Linear Map affecting a Hessian Matrix

Let $f:\mathbb{R}^2 \to \mathbb{R}$ be twice continuously differentiable and $L: \mathbb{R}^2 \to \mathbb{R}^2$ be linear described by the matrix $A \in \mathbb{R}^{2 \times 2}$. Let $g: \mathbb{R}^2 \...
1
vote
1answer
20 views

Motivation of Weird Functions for Hessian Matrix

Compute the Hessian matrix $H_f(0,0)$ of the following functions $f:\mathbb{R}^2 \to \mathbb{R}$: $f(x,y) = 1+x+y+ \left \langle \begin{pmatrix} x \\ y \end{pmatrix} , \begin{pmatrix} 1 & 1 \\ 1 &...
0
votes
1answer
33 views

Reordering the rows of a matrix and positive definiteness

I have two questions: 1) Does reordering the rows of a matrix affect its positive definiteness property? 2) If all elements of a non-symmetric matrix are positive, does this automatically imply ...
1
vote
1answer
28 views

Definition of convexity for multivariate functions

Let $f:\mathbb R^n \to \mathbb R$ be a twice differentiable function with $\mathbf{dom}\,f$ open. The Hessian of $f$ is defined as usual to be the matrix $\nabla^2f(x) \in \mathbb R^{n \times n}$ with ...
1
vote
0answers
35 views

Hessian on non-critical points

Hessian matrices can be used in the second partial derivative test to determine whether a stationary point is a maximum, a minimum, or a saddle. Can Hessians also be used to gain insight into non-...
3
votes
1answer
68 views

Hessian matrix computation for multi-layer neural networks (from Duda's book)

I am reading Duda's book, Section 6.9.1, about second-order methods for multi-layer neural networks. Considering the error criterion $J(\omega) = \frac{1}{2}\sum_{m=1}^{n}(t_{m}-z_{m})^{2}$, the ...
0
votes
0answers
43 views

Multivariable Newton-Raphson optimization with a Hessian

I am trying to use Newton-Raphson optimization to constrain some 2D black box function $ f(\mathbf{x}) = f(\alpha,\beta)$ I understand that the general update step proceeds as $$ \mathbf{x}_{n+1} = ...
6
votes
1answer
120 views

How do we define $D^2$ formally in differential geometry?

In this question @barto explains the definition of $Df|_p$ in differential geometry. This question is about the formal definition of $D^2f|_p$, given a function $f:M\to \mathbb R$ for some manifold $...
1
vote
2answers
53 views

Is it abuse of notation to write the hessian as $D^2f$?

The hessian matrix of a function $f:\mathbb R^n \to \mathbb R$ is often written as $D^2f=D(Df)$. However, as far as I know, $D$ is an operator that takes a function, and returns a matrix of functions (...
-1
votes
1answer
68 views

Prove that the function f is strictly convex [closed]

I came across this question when I self study my friends'note. He took data mining class in university and the question is like below and I cannot find answers from Google: I know that to prove that ...
0
votes
2answers
63 views

Does entry-wise non-negativity imply positive semidefiniteness?

Suppose we are given the function $f_1(x_1,x_2) = x_1x_2$, whose Hessian is $$\nabla^2 f_1(x)= \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$$ Now, it looks like this matrix is componentwise ...