Questions tagged [hessian-matrix]

The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both positive and negative eigenvalues then $x$ is a saddle point for $f$.

Filter by
Sorted by
Tagged with
-1
votes
0answers
7 views

What is the maximum of the function in this feasibility?

g(x,y)=xy2+x2−4xy. The region is only feasible if y≥−1, x≤3 and y≤x. I found the minimum in (2,2) but the other points were saddle points. Is there any maximum extrema in this region?
0
votes
0answers
26 views

Gradient descent with non-Lipschitz continuous gradients

In general, we know that for strongly convex functions for which we can compute the Hessian and find the Lipschitz constant $L$ of the gradient, gradient descent will converge provided that the step ...
0
votes
0answers
11 views

What do you call the block-diagonal entries of a hessian?

Consider some twice differentiable function $$f: \mathbb{R}^k \to \mathbb{R}$$ and some partition $k = m+n$, such that we may see this as a function $f(x_1,...x_m,y_1,....y_n)$. Is there some common ...
0
votes
1answer
21 views

Multivariable Function Optimization with Semidefinite Hessian Matrix

I'm writing to ask for support in carrying out an exercise about the optimization of a multivariable function. The function is: $f(x,y,z)=x^2+y^2z+z^2-2x$ Clearly the domain is $\mathbb{R}^3$, and $f$...
0
votes
1answer
30 views

Complexity of numerical derivation for general nonlinear functions [closed]

In classical optimization literature numerical derivation of functions is often mentioned to be a computationally expensive step. For example Quasi-Newton methods are presented as a method to avoid ...
1
vote
0answers
20 views

Guaranteed invertibility of the approximated Hessian in Levenberg-Marquardt

I need to show that the approximated Hessian in the Levenberg-Marquardt algorithm is guaranteed to be invertible, whereas in the Gauss-Newton algorithm, this is not always required to be true. ...
0
votes
1answer
34 views

Convexity analysis

I know that summation of two convex functions is also convex but I would like to know that does the same holds true for non-convex functions also such that the summation of two non-convex functions ...
0
votes
0answers
5 views

Showing that Lagrangian is concave using negative definiteness of the Hessian

I'm trying to solve a homework exercise where I am required to use the fact that, if a Lagrangian is concave, and a point $x^*$ satisfies the condition $\nabla \mathcal L (\mathbf x,\lambda)=0$, ...
0
votes
0answers
16 views

Avoiding inverse matrix calculation in Hessian approximation in Davidson-Flatcher-Powell method formula

My general concern is regarding denominator computation in this part $$\frac{s_ks_k^T}{y_k^Ts_k}\tag{1}$$ of the Hessian update in the method's algorithm/formula. Formula has taken from here : $$H_{...
2
votes
2answers
56 views

What is the Hessian matrix of $f\left(x\right)=\left\langle Ax,x\right\rangle \cdot\left\langle Bx,x\right\rangle $?

I'm trying to understand what is the Hessian matrix of $f\colon\mathbb{R}^{n}\to\mathbb{R}$ defined by $f\left(x\right)=\left\langle Ax,x\right\rangle \cdot\left\langle Bx,x\right\rangle $ where $A,B$ ...
0
votes
1answer
27 views

Convexity of Gaussian Q-function (monotone decreasing)

I have a known convex function. If I take the Gaussian Q-function of this convex function would the resultant function also be convex?
0
votes
0answers
13 views

If the Hessian is not symmetric, I am allowed to still use it for classification critical points?

I just thought to myself that if I have that the hessian will not be symmetric for using it for classification critical points? I know when the hessian is not symmetric so it means that the second ...
1
vote
1answer
28 views

Generalization of Gradient Using Jacobian, Hessian, Wronskian, and Laplacian?

I know there is a lot of topic regarding this on the internet, and trust me, I've googled it. But things are getting more and more confused for me. From my understanding, The gradient is the slope of ...
4
votes
2answers
81 views

Is there any obvious way to enforce a minimum amount of “positive definiteness” on a matrix?

Let $f(A,F)=\max(A,F)$ where $A\in\mathbb{R}$ is a variable and $F\in\mathbb{R}$ is a constant representing a "floor" below which the result should not be permitted to go. Is there any obvious ...
2
votes
0answers
45 views

Critical point and Hessian

Let $\phi:\mathbb{R^n}\to\mathbb R$ S.t $\phi(\vec{x})=\vec{x}^tA\vec{x}$ for $A\in M_n(\mathbb{R})$ ($A$ is doesn't have to be symetric). I need to show that $\vec{x_0} \in\mathbb{R^n}$ is a ...
0
votes
0answers
6 views

Computing tangent cone from hessian?

In optimization, one often assumes LICQ or other constraint qualification to determine the tangent cone of a set by takeing the gradient of the constraint functions. If the constraint has a vanishing ...
0
votes
1answer
19 views

region of positive definiteness of a hessian matrix

I am given a function f(x_1, x_2) = -cos(x_1) * cos(x_2/5). I need to identify the region in which the Hessian matrix is positive definite. I know that the first term of the Hessian matrix has to be ...
0
votes
1answer
32 views

Unconstrained optimization - parameters' values for global maximum

Consider the following function $f(x,y)=p \cdot x^a \cdot y^b - w_xx-w_yy$ with $(p,x,y,w_x,w_y) \in \mathbb{R^5_+} $ (this is because $f(x,y)$ is a profit function) (a) Solve the first-order ...
0
votes
2answers
41 views

Let $f(x,y)=\left(\frac{1}{2}x^{T}Qx\right)\left(\frac{1}{2}y^{T}Ry\right)$.given $g(x)=f(x,x)$ not convex.does $f$ convex?

I really need with this one: Let $$\begin{array}{c} Q,R\in\mathbb{R}^{n},Q,R\succ0\\ f:\mathbb{R}^{n}\times\mathbb{R}^{n}\longrightarrow\mathbb{R},g:\mathbb{R}^{n}\longrightarrow\mathbb{R}\\ f\left(...
0
votes
0answers
23 views

Hessian matrix (2x2) has trace,det=0 but a negative eigenvalue--is this matrix PSD?

I'm trying to show whether the function $f(x)=sin(\frac{\pi x_2} {2})$ is convex for $x\in \mathbb{R}^2$, where $0<x_1\le2$ and $0<x_2\le1$ When I compute the Hessian, I find $$H=\nabla^2f=\...
3
votes
4answers
67 views

Why do we need to determine the definiteness of the Hessian to decide what a critical point is?

In univariate calculus, if we know that $f'(c)=0$, we can determine if the function $f$ has a minimum at $c$ by checking that $f''(c) > 0$. The multivariate analogue of the second derivative is the ...
1
vote
1answer
39 views

How to quickly determine the definiteness of a large sparse matrix without using Sylvester's criterion?

I am currently trying to classify the stationary points of a function as either a maximum, minimum or saddle points based on the definiteness of the Hessian at those points. I have worked out that ...
2
votes
0answers
25 views

Simple example of a Jacobian of a matrix and/or Hessian of a vector

I wanted to add a comment to the following: Calculate the Hessian of a Vector Function Jacobian of a Matrix but I don't have enough reputation for that. Both answers are too short for me to ...
0
votes
0answers
9 views

Sufficient condition for optimality when hessian vanishes

If the first, second and third-order derivatives vanish at (x*,y*), how to derive the sufficient condition assuming 4th order derivative doesn't vanish?
0
votes
2answers
53 views

Gradient and Hessian of $\left(\frac{1}{2}\boldsymbol{x}^{T}Q\boldsymbol{x}\right)\left(\frac{1}{2}\boldsymbol{x}^{T}R\boldsymbol{x}\right)$

I've got a question about gradient and hessian of a scalar function: Let $Q, R \in \mathbb{R}^{n\times n}$ such that $Q, R \succ 0$. Let $g\left(\boldsymbol{x}\right) : \mathbb{R}^{n} \to \mathbb{R}$...
0
votes
1answer
49 views

Local stability of a min-max point

Suppose I have a smooth scalar function $f(x,y)$ where $x,y$ can be vectors. I am interested in finding a saddle-point: $$\min_x \max_y f(x,y)\qquad(1)$$ I have the intuition that at such a point, ...
0
votes
0answers
12 views

Hessian matrix for airfoil numerical optimization

In the case of airfoil optimization with conjugated gradient methods, I have to compute the the gradient of the objective function to obtain the conjugate direction. The objective function is given by ...
1
vote
0answers
36 views

Mistake in Rasmussen & Williams 2006 or my mistake?

I wonder if I make any stupid mistake understanding this, or can there actually a mistake in the bible of Gaussian Processes: Rasmussen & Williams 2006: Gaussian Processes for Machine Learning? ...
0
votes
2answers
30 views

Is the hessian of quadratic functions always symmetrical?

if we have: $$ f(x)= \frac{1}{2}x^\top A x - bx $$ then: $$ \nabla f(x) = Ax - b $$ and the hessian will be $A$. The function and its derivatives are all continuous, but $A$ (the hessian) can be non-...
0
votes
1answer
15 views

Exploit Hessian definiteness property in optimization

I would like to perform some minimization of a multivariate (most likely non-convex) function. Since I am capable of computing the gradient and the Hessian I like to apply Newton's method. Now, in ...
0
votes
0answers
25 views

2nd order approximation for nonlinear set of equations

I have two nonlinear functions $h_1 = f(x,y)$ and $h_2 = g(x,y)$ and I want to find an approximate solution for $x$ and $y$ as functions of $h_1$ and $h_2$. Fortunately, these functions also satisfy $ ...
0
votes
0answers
27 views

Hessian determinant equals 0

The question is to find all the critical points of the function, then determine whether each point is either a minimum, maximum or saddle point. The function is: $f(x,y) = x^3 - x^2 - x + y^3 - 2y^2$ ...
0
votes
0answers
22 views

Does there exist a real-valued function on the hyperbolic plane which has bounded hessian norm and unbounded gradient norm?

Does there exist a real-valued function on the hyperbolic plane which has bounded hessian norm and unbounded gradient norm? Specifically, consider the poincare half-plane model of the 2d hyperbolic ...
0
votes
0answers
12 views

Second-order sufficient optimality conditions

Suppose that $f$ : $ \Bbb R^n → \Bbb R$ is twice differentiable at x. Please show that x is a strict local minimum if $∇f($x$) = 0$ and the Hessian matrix H(x) is positive definite.
0
votes
4answers
59 views

Why is it that if $f(x) = x^T A x$ then $\nabla f = \frac{1}{2} (A+A^T) x$

I have that $$f(\vec{x},\vec{y}) = \vec{x}^T A \vec{x}$$ I have seen the result online that $$\nabla f = \frac{1}{2} (A+A^T) \vec{x}$$ yet I can't understand why this is the case. How do you get ...
0
votes
0answers
26 views

Is $(x_1,x_2) \mapsto \frac{x_1}{a-x_2}+b$ convex?

I encountered the following objective function: $$D(x_1,x_2) = \frac{x_1}{a-x_2}+b$$ Is it convex? If I use the Hessian to determine its convexity, I have $$H = \begin{bmatrix} 0 & 0 \...
1
vote
2answers
46 views

Can we interpret the determinant of a $2×2$ Hessian matrix in terms of the area of a parallelogram?

As we known, the determinant of a $2×2$ matrix is the area of a parallelogram? In this case, can we interpret the determinant of a $2×2$ Hessian matrix in terms of the area of a parallelogram? If so, ...
0
votes
1answer
23 views

Hessian of this function

We have $$f(w) = \ln(1+\exp(-w^Tx))$$ Here, $w$ and $x$ are $d \times 1$ What's the hessian of this function? I got gradient as $$\frac{-x\exp(-w^Tx)}{1+\exp(-w^Tx)}$$ then hessian as $$\frac{(1+\...
1
vote
1answer
63 views

Hessian matrix of the function defined with Implicit function theorem

Let $x=(x_1,...,x_n) \in \mathbb{R}^n, y\in \mathbb{R}$ and let $F(x,y)=F(x_1,...,x_n,y) \in C^2(\mathbb{R}^{n+1})$. Suppose we have all the hypothesis for the existence of the function $f(x)=y$ ...
0
votes
1answer
22 views

Is $(x,y,z) \mapsto \left(x^2 + \exp \left(y^2\right)\right)z$ convex?

I am trying to determine if the function $$f: \mathbb R^3 \to \mathbb R, \qquad (x,y,z) \mapsto \left(x^2 + \exp \left(y^2\right)\right)z$$ is convex. I tried to compute the Hessian matrix: $$\...
0
votes
0answers
16 views

Unconstrained function minimization when only given samples?

Let's say we want to minimize a function $f$ of the form: $$f(x) = \sum_{i=1}^n f(x_i)$$ where $x_i$ are samples. One way to solve these types of problem is via identifying a search direction via ...
0
votes
0answers
28 views

If i have an objective function with a lot of constraints. How can i prove conclusively that my problem is convex/ non-convex?

How to perform convexity analysis on a difficult objective function. I know about the Hessian matrix and Jensen's inequality. Both of them are difficult to derive in my case. What other theorems in ...
0
votes
1answer
16 views

Multivariable Calculus question, critical points and boundness.

I'm doing an exercise from the book Multivariable Real Analysis by Kolk and Duistermaat, however I'm not sure how to proceed after some point. It asks the following: we have $f:\mathbb{R}^{2}\...
0
votes
1answer
37 views

Hessian of a function with a symmetric matrix

I have seen the following function defined $ f(x,y) = \begin{bmatrix} x-x_0 \\ y-y_0 \end{bmatrix}^T A \begin{bmatrix} x-x_0 \\ y-y_0 \end{bmatrix}$ for a symmetric matrix A. It then states the ...
0
votes
1answer
27 views

If one of the leading second order principal minor of a Hessian matrix is negative can I claim that the matrix is negative semi-definite?

I am checking for Sylvester's criterion. If one principal minor is negative does it mean the Hessian is negative semi-definite?
3
votes
1answer
43 views

Estimates for the norm of Hessian

Let $u:\Omega \rightarrow \mathbb{R}$ a twice differential function, with $\Omega$ a subset of $\mathbb{R}^n$. Suppose that we have the following: $$D^2u\geq - \dfrac{(1+K^2)^{1/2}}{\epsilon}I$$ ...
0
votes
1answer
35 views

Hessian in second-order Taylor approximation

The second order Taylor approximation for a function $f(\mathbf{x}^*)$ is presented to me as $$f(\mathbf{x}^* + h\mathbf{y}) = f(\mathbf{x}^*) + h \nabla f(\mathbf{x}^*)^T \mathbf{y} + \dfrac{1}{2} ...
1
vote
0answers
20 views

Multidimensional integral with the Laplace method

Define $ \mathbb{W} := (W_{i, j})_{1 \leq i, j \leq k} $ and \begin{align*}%$ \varphi_{C, k}(\mathbb{W}) := C^2 \, \boldsymbol{1}_k^T (I_k + \mathbb{W} )^{-1} \boldsymbol{1}_k + \sum_{1 \leq i < j \...
1
vote
3answers
29 views

$\bar f(y) = f(Ty)$, how to compute the Hessian of $\bar f(y) $?

From Convex Optimization by Boyd and Vandenberghe: Let $T \in \Bbb R^{n \times n}$ be nonsingular. Let $f: \Bbb R^n \rightarrow \Bbb R$ convex and twice continuously differentiable. Define $\bar f(y) =...
0
votes
0answers
25 views

Hessian Matrix Inequality

Consider the linear elliptic operator of the form $$Lu=a^{ij}D_{ij}u+b^iD_iu$$ with coefficient satisfying $$\lambda I \leq [a^{ij}] \leq \lambda^{-1} I , \,\,\,\,\,\,\,\,\,\,\,\,\,\, |b^i|\leq \mu$$ ...

1
2 3 4 5
10