Questions tagged [hessian-matrix]

The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both positive and negative eigenvalues then $x$ is a saddle point for $f$.

Filter by
Sorted by
Tagged with
0
votes
1answer
49 views

How to find the Jacobian and Hessian of a function involving multiple Kronecker products?

I am having trouble finding the Jacobian and Hessian of this function involving the Kronecker product. I have a matrix $\mathbf{A}\in\mathbb{R}^{n\times n}$ and a vector $\mathbf{x}\in\mathbb{R}^{n^4}$...
-1
votes
0answers
39 views

Second order derivative of quadratic function

I try to find the second order derivative of the following equation w.r.t to $A$: $$J = -\frac{1}{2}(\textbf{x} - A)^T \textbf{C}^{-1}(\textbf{x} - A)$$ where $\textbf{x}_{2\times 1}$ is a vector and $...
0
votes
1answer
15 views

Convert function to convex on infinite range, knowing gradient and hessian

Let's take a function $f=sech(x)$ as an example. It is strongly convex in a limited range of $x$. It is further assumed that: $f$ is a "black box", $x$ as input ; output is the value of the ...
1
vote
1answer
24 views

Upper bound of the largest eigenvalue of $DAAD$.

$H = DAAD$, where $D$ is a diagonal matrix and the elements on the diagonal are real numbers (could be positive and negative). $A$ is positive semi-definite and can be diagonalized into $A=U^T\Lambda ...
0
votes
0answers
16 views

Non-negative second derivative of two-variable function implies convexity

I am trying to show that a function of two variables in $C^2$ is convex if its second derivative is non-negative. A twice differentiable function of several variables is convex iff its Hessian matrix ...
1
vote
0answers
126 views

How to implement Newton's method for optimization in a reliable way.

I'm trying to implement Newton's method for optimization. I want to find the local minima of a non-convex function (specifically a negative log likelihood) of $m$ variables where $m\sim 10$. Here is ...
-1
votes
0answers
25 views

Hessenberg matrix and eigenvalue problem

Let $A_{1} \in \mathbb{R}^{n \times n}$ be an unreduced Hessenberg matrix. Given $\mu$ (for simplicity, we assume that it is real), we compute the QR factorization $A_{1}-\mu I=Q_{1} R_{1}$ by Givens ...
0
votes
1answer
29 views

How to derive this form

can you please help me with the derivative of this function? I also need second derivative (need it for Hessian matrix) and I have no clue how to do it. Can you please help me? $f(x)=\frac{1}{4}(x^TQx)...
0
votes
1answer
29 views

Hessian of the log sum exp with affine input

Say I have this term $f(x) = log(\sum_{i=1}^{m} exp(K(a_i^Tx + b_i)))$, where $f: \mathbb{R}^d \rightarrow \mathbb{R}$, $x \in \mathbb{R}^d$, $A \in \mathbb{R}^{d \times m}$ and $a_i^T$ is column ...
1
vote
1answer
45 views

Problem with the continuous equivalent of Newton's method optimization

In the arcticle Fixed-Time Stable Gradient Flows: Applications to Continuous-Time Optimization I found an interesting formula and its properties. The screenshot of the page from the article I was led ...
0
votes
2answers
25 views

Hessian matrix for extremum [closed]

Is it enough to say a function doesn't have an extremum at a point if the determinant of the Hessian at that point is smaller than 0?
3
votes
1answer
54 views

Intuition: Convexity of Multivariate Functions and Positive Semidefiniteness of the Hessian

For a twice differentiable univariate function $f(x_1)$, the intuition why $f$ is convex if and only if $f''(x_1) \ge 0$ for all $x_1$ is pretty clear. At least once one has understood the notion of &...
0
votes
1answer
37 views

Is the trace of the Hessian matrix of the logistic loss function a convex function?

Consider the logistic loss function $$\ell(x, y,w) = \log \left( 1 + \exp \left(- y w^T x \right) \right)$$ where $x \in \Bbb R^d$ is an input sample and $y \in \{0,1\}$ is its label. We know that ...
2
votes
1answer
52 views

Find the extrema of $f(x,y) = x^3\cdot y^3$ in $\mathbb{R}^2$

I am asked to find the extrema of $$f(x,y) = x^3\cdot y^3$$ in $\mathbb{R}^2$ However, using the Hessian criteria, I get that the determinant of the Hessian matrix is zero for the two possible ...
0
votes
1answer
23 views

Problem understanding Quasi-Newton method: BFGS

I have problem understanding Quasi-Newton BFGS method. Quasi-Newton method is based on approximation of $f$ where: $f(x_k + \Delta x) \approx f(x_k) + \nabla f(x_k)^{\mathrm T} \,\Delta x + \frac{1}{2}...
0
votes
0answers
17 views

Negative curvature towards a certain direction

If $u^T\nabla^2 f(z)u$ is strictly negative, can we say that the Hessian $\nabla^2 f(z)$ has at least one negative eigenvalue corresponding to $u$?
0
votes
0answers
21 views

How to compute the Hessian of a vector valued function?

The best learning rate of a cost function is strictly less than 2λ, where λ is the largest eigenvalue of the Hessian. I want to get the best learning rate of my gradient descent algorithm, which is: <...
0
votes
1answer
45 views

Polynomial with small hessian determinant

Let $P:\mathbb R^2\to \mathbb R$ be a polynomial with no constant or linear terms, and with all other coefficients bounded by 1 in absolute value. Suppose its Hessian determinant is equal to $\epsilon ...
0
votes
0answers
7 views

Interpretation of error between Hessian approximation and real Hessian - Quasi-Newton Method

$$ ||I- H_{k}^{BFGS}\nabla^{2}f(x_{k})||_{2}$$ , where $H_{k}$ is the inverse of hessian approximation at each iteration. I am given this expression to assess the error in Hessian approximation in ...
1
vote
1answer
88 views

How to calculate the inverse of Hessian matrix?

In the question Calculating the determinant of the Hessian of a function, we know that the Hessian of function $$f(x_1,\dots,x_n) = g\bigg(x_1,\bigg(\sum_{i=2}^n x_i^2\bigg)^{1/2}\bigg), $$ and denote ...
0
votes
0answers
22 views

Does the minimum of a function with an indefinite Hessian matrix lie on the boundary

I have a function $f(\boldsymbol\gamma)$ where $\gamma_i \in [0,1]$ for $i \in \{1,\ldots, N\}$. The Hessian matrix $H$ for this function is symmetric, the diagonal $h_{i,i} = 0$ and all values $h_{i,...
0
votes
1answer
65 views

On “the Hessian is the Jacobian of the gradient”

According to Wikipedia, The Hessian matrix of a function $f$ is the Jacobian matrix of the gradient of the function $f$; that is: $H(f(x)) = J(\nabla f(x))$. Suppose $f : \Bbb R^m \to \Bbb R^n,x \...
2
votes
2answers
97 views

I need a limit definition for the Hessian, does this work?

Let $f: \mathbb{R}^n \to \mathbb{R}$ be of class $C^2$. Let $x$ be a non-degenerate critical point of $f$. Prove that there is an open neighborhood of $x$ which contains no other critical points of $f$...
1
vote
0answers
41 views

Is $f(x, y) = xy-2x-y$ an hyperbolic paraboloid?

so I solved the function, finding a hyperbolic paraboloid with a saddle point $xy-2x-y=-2$ at $(x, y)=(1, 2)$, with no global or local maxima and/or minima. But the answer sheet marks that it indeed ...
0
votes
1answer
30 views

Sharpest upper bound on $|f(x)|$ using multivariable Taylor approximation, Hessian, or any other method

(Based on a question in University of Pittsburgh's Preliminary examination: Aug 2016, question 5.) Let $f\colon \mathbb{R}^2 \to \mathbb{R}$ be a $C^2$ function. Let $B=\{x \in \mathbb{R}^2\colon |x|\...
0
votes
0answers
17 views

Is the following function convex or nonconvex?

Suppose $\mathbf x=\left(\mathbf x_1,\mathbf x_2\right)^T$. Is the function $f(\mathbf x)=\|\mathbf x_1\mathbf x_1^H+\mathbf A\mathbf x_2\mathbf x_2^H\mathbf A^H\|^2$ convex with respect to $\mathbf x$...
0
votes
0answers
45 views

Matrix “divided” by a matrix. Can this be done?

So I am working on a problem and which basically looks like this $- \frac{x^t A^{-1}x}{M x^TA^{-2}x}$ where $M$ is a scalar, $x$ is a vector and $A$ is a matrix (which in the actual problem is the ...
0
votes
0answers
22 views

Explanation for dimension mismatch of Hessian of $f(x) = \log(1+x^TQx)$ where $Q=Q^T$ and $Q$ is positive-definite

According to this link, we have that given $f(x) = \log(1+x^TQx)$, the Hessian matrix is given by $$\nabla^2f(x)=\frac{Q^T+Q}{1+x^TQx}- \left( \frac{(Q+Q^T)x}{1+x^TQx}\right)^T \left( \frac{(Q+Q^T)x}{...
1
vote
0answers
43 views

Bound eigenvalues of $D^\top C D$, for $C$ positive semidefinite

I need to bound the eigenvalues of a matrix of the form $D^\top C D$, where: $D \in \mathbb{R}^{n^2 \times n(n-1)/2}$ is a duplication matrix, in particular the matrix for which $D \operatorname{...
0
votes
0answers
14 views

Function of several variables whose hessian is a Hankel matrix

I am studying a function $f:\mathbb{R}^n\rightarrow\mathbb{R}$, whose Hessian $H_{i,j} = \frac{\partial^2 f}{\partial x_i \partial x_j}$ has Hankel symmetry, namely $H_{i,j} = h_{i+j}$. This is true ...
1
vote
1answer
27 views

Hessian of $f(x) = e^{u^Tx}$ for some $u\in \mathbb{R^d}$

I'm seeing the hessian is $uu^Te^{u^Tx}$. From my understanding since this is a single variable function, we can basically treat $u^T$ as a constant and the hessian is equivalent to the second ...
1
vote
2answers
61 views

Is the negative normalized entropy convex?

The negative normalized entropy is defined as $$h:\mathbb{R}_{>0}^n \rightarrow \mathbb{R} \ , \ h(x)=\sum_{i=1}^n x_i\log \frac{x_i}{\sum_{j=1}^n x_j} \ .$$ Is this function convex? Its Hessian is ...
0
votes
0answers
52 views

Possibility of confusion caused by the use of $\nabla$

This is a question about notation. Of course, as long as the notation is clearly defined, it doesn't matter at all which notation we use, but it's still helpful to ask about a few possible confusions ...
1
vote
0answers
31 views

Is it possible to calculate the feed forward Hessian inverse?

Did I made a calculation error? Say we had a simple one layer perceptron where: $f$ is the activation function, $w$ is the weights matrix, $b$ is the bias vector, $x$ is the input vector, $y$ is the ...
1
vote
1answer
49 views

Hessian and metric transformation

I am trying to obtain the relation between the Hessians when I consider the transformation $\tilde{g}=\sigma^{-1}g$, for $(M^n,g)$ semi-Riemannian manifold and $f\in C^\infty(M)$, then if $$\text{Hess}...
0
votes
0answers
42 views

Is $g(x)=f(Ax)$ strongly convex if $f(z)$ is strongly convex?

Suppose that f(z) has gradient $\nabla f(z)$ and hessian h(f(z)). Let g(x)=f(Ax). Suppose that f(z) is strongly convex. Is g(x) necessarily strongly convex? If so, prove, if not find a counterexample....
2
votes
0answers
34 views

Compute this Hessian

Let $f$ be a smooth, real valued function on a complete Riemannian manifold $(M,g)$ and $x_0\in M$. Define the function $\tilde f=f\circ Exp_{x_0}:T_{x_0}M\to \mathbb R$. What is the Hessian of $\...
1
vote
2answers
53 views

Gradient and Hessian of a matrix

For $g(y) = f(D^{\frac{1}{2}}y)$ where $D^{\frac{1}{2}}$ is a matrix to the power half and $ x = D^{\frac{1}{2}}y$ Then $\nabla g(y) = \nabla f(D^{\frac{1}{2}}y) = D^{\frac{1}{2}} \nabla f(D^{\frac{1}{...
0
votes
2answers
26 views

How to expand this tensorial Taylor expansion to the $n$th term?

Wikipedia describes the use of the Hessian matrix in a Taylor series expension. I've noted that the first term is written in terms of the original function, the second term uses the gradient of the ...
1
vote
0answers
26 views

Geometric intuition for Hessian matrices

I have been learning about multivariate calculus on my own. I have learned, notation-wise the properties of concave and quasi-concave functions. But I am finding it very difficult to find geometric ...
0
votes
0answers
23 views

How to compute the Gradient and the Hessian matrix of $\sum_{m=1}^M log (1 + \exp(- a_m^T x))$?

I am trying to find out the Gradient and the Hessian of the function $ f(\mathbf{x})=\sum_{m=1}^M log (1 + \exp(- \mathbf{a_m}^T \mathbf{x}))$, where all of $\mathbf{a_1},...., \mathbf{a_M} \in \...
0
votes
1answer
35 views

Why has the kernel of the 2x2 hessian 3 linear functions?

I have calculated something characteristic polynome and such things but i think thats the wrong way to see this. My try is : On $\mathbb{R}^2$ the Hessian matrix can be relaized as $$ \mathbf{H}_\...
1
vote
0answers
11 views

Expression on the computation of a Fisher matrix element : different cases

I remind the context : The aim of the analysis presented here is to obtain estimates on cosmological parameter measurements, i.e. the posterior distribution $P(\theta \mid x)$ of the vector of (model) ...
1
vote
1answer
47 views

Showing that a general Hessian matrix is positive semidefinite

Given vector $a \in \mathbb{R}^n$, show that the scalar field $g : \mathbb{R}^n \to \mathbb{R}$ defined by $$g(\mathbf{x}) = -\log ( f(\mathbf{x}))$$ where $$f(\mathbf{x}) = \dfrac{1}{1+\exp(-a^T\...
0
votes
1answer
35 views

Let $f: \Bbb R^n\to\Bbb R$, the Hessian of $f$ is postive definite everywhere. Show that $\mathrm{grad} \, f: \Bbb R^n\to\Bbb R^n$ is bijective.

Let $f: \Bbb R^n\to\Bbb R$, the Hessian of $f$ is postive definite everywhere. Show that $\mathrm{gradient} \, f: \Bbb R^n\to\Bbb R^n$ is bijective. Argue by contradtion, if $\mathrm{grad} \, f(x_1)=\...
0
votes
1answer
27 views

Gradient, Hessian, and minimum of a vector function?

Given the function $f(\vec{x})=(\frac{1}{2})\vec{x}^TP^TP\vec{x}+q^T\vec{x}+r$, where $\vec{x}$ and $q \in \mathbb{R}^n$, and $P \in \mathbb{R}^{n x n}$ is full rank and $r \in \mathbb{R}$, I am ...
1
vote
1answer
65 views

Determining the convexity of functions

I am trying to show the convexity of functions which include exponential variables with different powers. My question is, for quadratic functions($x^TQx$) it is easier to show as showing Q is ...
2
votes
0answers
41 views

$p$ is an inflection point if and only if the Hessian $\mathcal{H}_{P}$ vanishes at $p$

I'm trying to comprehend the proof of the following proposition. Let $C$ be a projective plane curve over $\mathbb{C}$ and let $p$ be a smooth point of $C .$ Let $P$ be a homogeneous polynomial in $\...
1
vote
0answers
37 views

How to find Hessian matrix by applying the jacobian on the gradient.

Let $f(x) = (a^Tx+\alpha(b^Tx)^2+\beta)^2$ where $a, b, x \in \mathbb{R}^n$. How can one calculate the Hessian matrix using the fact that $H(f(x)) = (J_x(\nabla f(x)))^T$ where $J_x$ is defined as the ...
-1
votes
1answer
45 views

Matrix with positive and negative eigenvalues (proof/saddle point)

Let $f:\mathbb{R}^n \to \mathbb{R}$ with $f(\textbf{x})=\frac{1}{2}\textbf{x}^T\textbf{B}\textbf{x}$ where $\textbf{B} \in \mathbb{R}^{2\times 2}$ has one positive and one negative eigenvalue for the ...

1
2 3 4 5
13