Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hessian-matrix]

The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both ...

1
vote
0answers
12 views

Approximation of Inverse Hessian or Inverse Hessian Square Root times a vector

I know there are good methods for approximating a Hessian times a vector without actually forming the hessian. (Example here). Are there any methods of approximating the Inverse of hessian times a ...
0
votes
1answer
13 views

Calculating the Hessian of a function $f :\mathbb{R}^{n} \rightarrow \mathbb{R}$ given an expression for the gradient including eigenvalues

We are given a function $f :\mathbb{R}^{n} \rightarrow \mathbb{R}$ such that: $\nabla{f}(x) = \lambda x$ where $\lambda$ is a scalar, and an eigenvalue of some Matrix $A \in M_{n \times n}$ that ...
1
vote
1answer
31 views

Calculating the Hessian of a function $f :\mathbb{R}^{n} \rightarrow \mathbb{R}$ given the gradient

We are given a function $f :\mathbb{R}^{n} \rightarrow \mathbb{R}$ such that: $\nabla{f}(x) = Px$ where $P \in M_{n \times n}$ is an $n \times n$ matrix, and $x \in \mathbb{R}^{n}$. What is the ...
0
votes
1answer
45 views

Matrix calculus : Derivative with respect to two vectors

I have a function of two variables, $\vec{x_i} \in \mathbb{R}^n $ and $\vec{y_j} \in \mathbb{R}^n$: $f(\vec{x_i},\vec{y_j}) =\sum_\limits{i,j} \left( \vec{x_i} \cdot \vec{y_j} + z_{ij} \right)^2$ I ...
1
vote
1answer
71 views

Number of Solutions to the Problem: Minimize $|| A - Afc^T / c^T f ||^2$ such that $\sum_i c_i = 1$

Let $A$ be a $M$ by $N$ matrix, $f$ is a column matrix with $N$ elements, $c$ is a column vector with $N$ elements that I need to solve for, and $|| \cdot ||$ is the Frobenius norm of the matrix. I ...
0
votes
1answer
60 views

Hessian wrt. MATRIX-VARIABLE for a Quadratic Inner Product.

Given standard matrix inner product, \begin{equation} \begin{aligned} f(\textbf{X}) := & \;\;\;\; {\langle}{\textbf{X}, \textbf{A}\textbf{X}}{\rangle}\\ =& \; \text{tr} (\textbf{X}^{\text{...
0
votes
1answer
26 views

At a Critical Point the Hessian Equals the Frst Nonconstant Term in the Taylor Series of $f$?

My textbook defines Hessian functions as follows: Suppose that $f: U \subset \mathbb{R}^n \to \mathbb{R}$ has second-order continuous derivatives $\dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\...
0
votes
0answers
19 views

hessian calculation with division by zero for first derivative

I am wanting to calculate the hessian from an example dataset that keeps evolving. At one particular data set, one of the parameters (z) does not change, which causes a division by zero for the ...
0
votes
0answers
17 views

Hessian matrix and analysis

I've been trying to solve this problem for a long time, but I don't know exactly how to start: Translation: M is square dxd, we want to show that f belongs to C^2 in R^d. After we want to calculate ...
0
votes
1answer
52 views

Hessian of negative log-likelihood of logistic regression is positive definite?

I'm trying to show that the Hessian of the negative of the log likelihood with two parameters is positive definite, but I'm not sure how to go about it once I compute the Hessian. The function is: $-...
0
votes
0answers
14 views

Hessian Matrix Denominator

Isn't the denominator for each diagonal term in the Hessian matrix as given on Wikipedia incorrect? Hessian Matrix Instead of the denominator being $\partial x_1^2 $, shouldn't it be $\partial^2 x_1 ...
0
votes
1answer
47 views

Is $z = x^2y^3(1-x-y)$ convex or concave?

Is there some kind of trick to defining the domain of the concavity/convexity (if it exists)? I have no idea how to work with the resultant hessian
2
votes
1answer
44 views

gradient and hessian of $e^{x^Tx}$

I want to find gradient and hessian of $e^{x^Tx}$ My attempt: $\nabla = 2x^Te^{x^Tx}$ Hessian $= 2e^{x^Tx}I + 2xx^Te^{x^Tx}$ Is that correct?
-1
votes
0answers
21 views

calculate hessian matrix in markov random field

i try to learn markov random field parameters , for this i want to calculate the hessian of the probability function, i calculate the hessian in 2 way and got two different answer and i cant ...
0
votes
0answers
32 views

Regular surface with Curvature mean constant

Let $F:R^3->R$ given by $F(x,y,z)=ax^2+by^2+cz^2 - 1$ I wanto to proof that inverse image of F apply at point $0$ is a regular surface with curvature mean constant. Let me tell you what my first ...
0
votes
0answers
16 views

Interpretation of signs and magnitudes of eigenvalues of Hessian

Suppose I have a 50-dimensional field. I compute the Hessian matrix at a stationary point and find 40 negative eigenvalues + 10 positive ones. Can I conclude that the point is "mostly" a maximum with ...
0
votes
1answer
25 views

Sufficient Conditions for quasiconcavity proof

I was reading a book and it says that the sufficient condition for a function to be quasiconcave is that its Bordered Hessian matrix is negative definite. I can't seem to understand this. Please help! ...
0
votes
1answer
24 views

Hessian and gradient in a matrix

There is following matrix: $$\begin{pmatrix}\nabla^2g(x) & \nabla g(x)\\\ \nabla g(x)^{T} & 1\end{pmatrix} \ge 0$$ the "$\ge$" is general inequality (not element-wise), meaning the matrix ...
0
votes
1answer
50 views

Do you agree that the following domain is not convex?

The domain is given as $x_1,x_2,y_1,y_2 \in \mathbb{R}$ with: $$x_1-y_1^2-4 \geq0,\quad x_2-y_2^2-4 \geq 0, \quad x_1\leq 10 ,\quad x_2 \leq 10 $$ We must prove this is convex. This is my approach: I ...
2
votes
0answers
55 views

A question regarding convex function and Hessian

I came across with the problem which asks me to show that: $f:\mathbb{R}^n\rightarrow \mathbb{R}, f\in C^2$ with Hessian of $f$ well-defined on $\mathbb{R}^n$, is convex if and only if $\nabla^2f\...
0
votes
1answer
26 views

Convexity and concavity with Hessian matrices

I want to analyze two Hessian matrices regarding definiteness to formulate conclusions whether the functions are convex or concave. If you could check my thoughts, I’d be grateful. $H_1(x)=\left ( \...
0
votes
1answer
20 views

Find and classify the stationary points for $f(x,y) = (4x_1^2 - x_2)^2$

So first I calculated the gradient which was $\nabla f(x) = (64x_1^3 - 16x_1x_2, -8x_1^2 + 2x_2)^T.$ Then setting the equations in this equal to $0$ I solved for $x_2$ and got $x_2=4x_1^2$. Then ...
0
votes
2answers
37 views

Clarification of Textbook Explanation of Hessian Matrix, Directional Second Derivative, and Eigenvalues/Eigenvectors

My machine learning textbook has the following section on the Hessian matrix: When our function has multiple input dimensions, there are many second derivatives. These derivatives can be collected ...
3
votes
1answer
53 views

Hessian of a $C^{\infty}$ function $f: \mathbb R^n \to \mathbb R$ restricted to a subspace

Let $f: \mathbb R^n \to \mathbb R$ be an infinitely differentiable function. Let $\mathcal V \subset \mathbb R^n$ be a vector subspace. Let $g = f|_{\mathcal V}$ be the restriction of $f$. Then we can ...
0
votes
0answers
31 views

Hessian decomposition?

I came across the decomposition below but I am not sure how to derive it or if it is in fact true. Can anyone share some insight? $D^2v=\partial^2_{nn}v(n^T)+\partial^2_{nr}v (nr^T+rn^T)+\partial^2_{...
1
vote
0answers
26 views

How to classify the degenerate critical (stationary) points of a multivariate function?

I have a multivariate function in one of whose critical points the Hessian matrix is singular. Is there any general method to determine the type of this critical point? Would be worth plotting the ...
0
votes
1answer
27 views

what if the hessian of the Lagrangian is null?

In a non-linear optimization problem, suppose given a candidate solution $x*$, we want to verify second order sufficient conditions of optimality but the Hessian of the Lagrangian of the problem at $x*...
4
votes
0answers
35 views

A matrix problem about egienvalue and trace

Consider an $m\times m$ positive definite and Hermitian matrix $\mathbf{M}$ and an arbitrary $m\times n (m>n)$ para-unitary matrix $\mathbf{R}$, i.e., $\mathbf{R}^H\mathbf{R}=\mathbf{I}_n$. ...
0
votes
2answers
33 views

low-complexity matrix inversion algorithm for a near-identity matrix?

As I know, the general complexity of matrix inversion is $O(n^3)$, but it is a little bit high. My matrix is $(I + A)$ , where $I$ is an $n \times n$ identity matrix and $A$ is a hermitian matrix ...
4
votes
3answers
96 views

Gradient and Hessian of $x x^T$ w.r.t. $x$, where $x \in \mathbb{R}^{n \times 1}$,?

Question: Can we find the gradient and Hessian of $x x^T$ w.r.t. $x$, where $x \in \mathbb{R}^{n \times 1}$ ? EDIT: If we can, may I know how to compute that? Thank you.
1
vote
0answers
33 views

Condition number of Hessian matrix when Hessian is singular

In gradient descent on a quadratic problem, $$\min _{x \in \mathbb{R}^n}\frac12 \langle Hx, x \rangle + \langle b, x \rangle + c \qquad H \text{ symmetric, positive semi-definite}\quad b \in \...
3
votes
2answers
97 views

Calculate the Hessian of a Vector Function

I'm working with optimisation. I am trying to obtain the hessian of a vector function: $$ \mathbf{F(X) = 0} \quad \text{or} \quad \begin{cases} f_1(x_1,x_2,\dotsc,x_n) = 0,\\ f_2(x_1,x_2,\...
0
votes
2answers
67 views

Example of convex function which is differentiable, but not twice differentiable?

Are there convex functions for which hessian is not defined, but the gradient is defined everywhere? I was looking at projected gradient descent, as well as Newton's method for solving optimization ...
0
votes
1answer
28 views

Second directional derivative and Hessian matrix: a clarification on the proof

I have been reading this answer and I couldn't get the proof. The argument is about how we can write the second directional derivative of $f:\mathbf{R}^m\rightarrow\mathbf{R}$ with respect to ...
3
votes
1answer
31 views

How to check for local extrema or saddle point given an semidefinite matrix

I've computed the Hessian of a given function $f(a,b,c) = y-a\sin(bx-c)$ and got the following result: $\begin{pmatrix} 0 & -x\cdot\cos(bx - c) & \cos(bx - c) \\ -x\cdot\cos(bx - c) & ax^...
2
votes
3answers
140 views

What is the higher-order derivative test in multivariable calculus?

In single-variable calculus, the second-derivative test states that if $x$ is a real number such that $f'(x)=0$, then: If $f''(x)>0$, then $f$ has a local minimum at $x$. If $f''(x)<0$, then $...
1
vote
0answers
49 views

Hessian matrix vs differential 2-form

Could someone clarify the convention that the second derivative of a scalar function $f: \Bbb R^n \rightarrow \Bbb R$ is sometimes defined as a linear operator $D^2f : \Bbb R^n \rightarrow L(\Bbb R^n, ...
2
votes
5answers
213 views

Hessian on linear least squares problem

I tried to calculate the Hessian matrix of linear least squares problem (L-2 norm), in particular: $$f(x) = \|AX - B \|_2$$ where $f:{\rm I\!R}^{11\times 2}\rightarrow {\rm I\!R}$ Can someone help ...
2
votes
0answers
65 views

Sign of the Bakry-Emery curvature operator $\Gamma_2(f) :=|Hess f|^2+Ric(\nabla f,\nabla f)$

I am not an expert of Riemannian geometry (coming mainly from functional analysis in $\mathbb R^n$). Let $(M,g)$ be a compact $n$-dimensional Riemannian manifold with Ricci curvature bounded from ...
0
votes
0answers
30 views

Approximating Hessian with BFGS for a matrix variable

So as we know the approximation of the inverse of the hessian matrix using the BFGS method is calculated with the following formulas : $$q_{k+1} = (I-p_k s_k (y_k)^T)q_k(I- p_k y_k (s_k)^T) + p_k ...
0
votes
0answers
33 views

Reading Help - Hessian, Linear Algebra, Finding Eigenvalues

Edit: I've cut some parts to try and focus this question on linear algebra I'm having a lot of difficulty understanding some parts of this paper, which I'm reading out of personal interest, yet am ...
1
vote
1answer
34 views

When does $f(x_0)$ maximum $\implies \det \mathrm{Hess}_f(x_0)\neq 0$?

Let $f:\mathbb{R}^n\to\mathbb{R}$ be a smooth function and let $x_0\in \mathbb{R}^{n+1}$ be a critical point of $f$ such that $f(x_0)$ is a local maximum. Question: Under which conditions does it ...
0
votes
1answer
39 views

Local minimum of an analytic function

This is a follow-up to a previous question of mine. I know that any local minimum $x_0$ of a function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ has positive semi-definite Hessian $H(x_0) \succeq 0$. ...
3
votes
2answers
38 views

Local minimum has neighbourhood with positive semi-definite Hessian?

I know that any local minimum $x_0$ of a (twice continuously differentiable) function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ has positive semi-definite Hessian $H(x_0) \succeq 0$. Can we say ...
0
votes
1answer
15 views

Properties of norms of inverse Hessians near a solution

In theorem 3.5 of the book on Numerical Optimization by Jorge Nocedal and Stephen J Wright, Second edition, is given the proof for the quadratic convergence to the solution of a sequence of iterates ...
0
votes
1answer
39 views

Finding the extrema of $f(x,y)$ when ALL critical points $=0$

Let $f: \mathbb R^{2}\to\mathbb R$ and $f(x,y)=(x^2-1)(y-1)^2$. Find all the extrema and saddle points. Idea: Setting $\nabla f(x,y)=0$, we get: i) $2x(y-1)^2=0$ ii) $(x^2-1)2(y-1)=0$ Therefore ...
0
votes
0answers
39 views

Convex inequality is twice differentiable with Hessian matrix

Let $f: A\subseteq \mathbb{R}^n \rightarrow \mathbb{R}$ be continuously differentiable such that $$\nabla^T f:A\rightarrow\mathbb{R}^n, x\mapsto [\delta_1 f(x),\dots,\delta_n f(x)]^T$$ is continuously ...
0
votes
1answer
20 views

Evaluating an extremal value if the hessian matrix has at least one eigenvalue which is zero

$$f(x,y) = 2x^4-3x^2y + y^2$$ We want to find all extremal values: $$df(x,y)=(8x^3-xy,-3x^2+2y)\overset{!}{=}0 \quad \Rightarrow \quad p=(0,0)$$ $$H_f(x,y)=\begin{pmatrix}24x^2-6y& -6x\\-6x &...
-1
votes
2answers
42 views

Extrema when Hessian Matrix is $0$ at $(0,2)$

Let $f: \mathbb R^{2}\to \mathbb R$, $f(x,y)=xy^2-4xy+x^4$ Find all critical points of $f$ and determine whether or not they are extrema and their type: Setting $0=\nabla f(x,y)$ I get the ...
0
votes
0answers
89 views

Interpretation of the magnitude of an eigenvalue?

We know that the eigenvalues of a Hessian matrix can provide information about the curvature of a function under study. Specifically, we know that the sign of the eigenvalues give us information ...