Questions tagged [hessian-matrix]

The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both positive and negative eigenvalues then $x$ is a saddle point for $f$.

Filter by
Sorted by
Tagged with
0 votes
1 answer
9 views

Laplacian in terms of integral of Hessian over unit sphere

Let $u \in C^2$. Somehow I tend to believe that the following identity $$ \int_{\partial B_1 (x)} \big\langle (\nabla^2 u) (x) y, y \big\rangle d\sigma_y = C \Delta u(x) $$ holds for some constant $C&...
user avatar
  • 346
1 vote
1 answer
39 views

Second partial derivative: What happens if $f_{xy}^2$ is larger than $f_{xx} f_{yy}$

Considering $(a,b)$ is a critical point of a funciton $f(x,y)$ and $D(x,y) = det(H(f(x,y))) =f_{xx}(x,y)f_{yy}(x,y) - (f_{xy}(x,y))^{2}$ is the determinant of the hessian matrix from that function. If ...
user avatar
-1 votes
0 answers
45 views
+50

Numerical computation of Hessian matrix

I'm following the book Numerical Recipes: The Art of Scientific Computing, Third Edition (2007) where in chapter "5.7 Numerical Derivatives" the formula for computing the second derivative (...
user avatar
  • 237
0 votes
0 answers
12 views

Which derivatives symmetrizes the Hessian matrix?

The Hessian matrix is a table of repeated derivatives. For some functions it is asymmetric. This seems to depend on the type of derivative being used. Which derivatives are suitable to make the ...
user avatar
0 votes
1 answer
64 views

Does the Hessian matrix of energy function of a gradient system have to be positive semidefinite when the system has one globally stable point?

Given a gradient dynamical system $$\frac{d\theta_i}{dt}=f_i(\theta_1,\cdots,\theta_n),\forall i\in\{1,\cdots,n\},$$ where $$\frac{\partial G}{\partial \theta_i}=f_i(\theta_1,\cdots,\theta_n),$$ where ...
user avatar
0 votes
0 answers
18 views

How to check whether this matrix is positive semidefinite?

Given the following symbolic matrix \begin{equation*} A= \begin{pmatrix} -\cos(\theta_1-\theta_2)&\cos(\theta_1-\theta_2)&0 \\ \cos(\theta_1-\theta_2) & -\cos(\theta_1-\theta_2)-\cos(\...
user avatar
1 vote
0 answers
31 views

Why is $\mathbf{u}=\alpha\ \mathbf{e}_k$?

I've read this answer to a question about Symmetric Rank Update (SR1). In this approach, we require the update of the Hessian matrix to be of the form $$ \mathbf{B}_{k+1}=\mathbf{B}_{k} + \mathbf{u}\...
user avatar
  • 1,406
-1 votes
0 answers
21 views

Problem With Deep Learning Book By Aaron Courville, Ian Goodfellow, and Yoshua Bengio

Hello Everyone, I am reading Deep Learning Book By Aaron Courville, Ian Goodfellow, and Yoshua Bengio. In the Numerical consumption chapter, section 4.3.1 got a problem, There were a linked, We can ...
user avatar
0 votes
1 answer
32 views

Strongly convexity of a loss function

I want to calculate the strongly convex parameter $\sigma$ for this loss function: $$ l_Z(Z)=||Z-A||^2_F+\lambda tr[Z^TBZ] $$ where $Z\in \mathcal{R}^{n\times m}$, the value of $A,B$ and $\lambda$ are ...
user avatar
  • 193
1 vote
1 answer
42 views

Find gradient and Hessian for optimization problem

Given $$S_\mu(u) := \sum_{i=1}^n \sqrt{u_i^2+\mu^2}-\mu$$ a smoothed $1$-norm. Using Newton's method, calculate $$\min_{u\in \mathbb{R}^n}\frac{1}{2} \|u-u_0\|_2^2 + \alpha S_\mu(\nabla u)$$ where $$(\...
user avatar
6 votes
1 answer
89 views

$\sum_{i,j=1}^nx_ix_j\frac{\partial^2f}{\partial x_i\partial x_j}=0$ and $\nabla f(0)=0$ implies constancy of $f$ in $B_1(0)$

Let $B_1(0)$ be the unit ball in $\mathbb R^n$ centered at the origin. Assume that the function $f\in C^2(B_1(0))$. Prove that $1)$If $f$ satisfies $$\sum_{i,j=1}^nx_ix_j\frac{\partial^2f}{\partial ...
user avatar
  • 804
0 votes
1 answer
28 views

Hessian in abstract notation

For functions $u$ and $f$, is $$\mathrm{Hess} f(\nabla u,\nabla u)=(\nabla_i\nabla_j f)(\nabla^i u)(\nabla^j u),$$ or is it $(\nabla_i\nabla_j f(\nabla^j u))(\nabla^i u)$?
user avatar
1 vote
0 answers
33 views

How to determine the type of saddle points of a surface? [closed]

Let $f(x,y)$ be a real polynomial with the variables $x, y$. Is there an easy way to determine the type of saddle points of the surface $z=f(x,y)$? That is, if they are ordinary saddle points (like ...
user avatar
0 votes
1 answer
105 views

Why does $\|\nabla f(x) - \nabla f(y)\| \leq L \|x-y \|$ imply $\| \nabla^2 f(x) \| \leq L$?

Let $f$ a twice continuously differentiable function. Also let $\nabla f$ be Lipschitz continuous with Lipschitz constant $L$, i.e., $$\|\nabla f(x) - \nabla f(y)\| \leq L \|x-y \|.\tag{1}$$ On page ...
user avatar
  • 721
1 vote
1 answer
32 views

Hessian of quadratic objective function

I have the quadratic function $$f(\boldsymbol{x}) = \frac{3}{2} \left (x_{1}^{2}+x_{2}^{2} \right) + (1+a) x_{1} x_{2} - \left(x_{1} + x_{2} \right) + b$$ where $a, b \in \Bbb R$ are unknown ...
user avatar
  • 133
2 votes
2 answers
59 views

Can every symmetric Jacobian matrix be a Hessian matrix?

Say we have a function $\mathbf{f}(\mathbf{x})$ where $\mathbf{x}\in\mathbb{R}^n$ and $\mathbf{f}:\mathbb{R}^n\rightarrow\mathbb{R}^n$ with a Jacobian matrix $\mathbf{J} = \partial \mathbf{f}/\partial ...
user avatar
  • 671
0 votes
0 answers
58 views

Compute Lipschitz constant of Hessian of logistic loss

Convergence theory of Newton's method assumes that the Hessian $\nabla^2 l(w)$ of the loss function l is Lipschitz continuous, i.e. $\big\Vert \nabla^2 l(w_1) - \nabla^2 l(w_2)\big\Vert_2 \leq L \big\...
user avatar
1 vote
1 answer
35 views

What do you call third order derivative matrix and what does it geometrically signify?

The first-order derivatives matrix is known as Jacobian, gives the gradient of the graph. Similarly, the second-order derivatives matrix is Hessian, which gives the curvature of the plot. What next? i....
user avatar
1 vote
0 answers
35 views

Why is $f(x,y) := 6y+4x-x^3y$ infinitely often continuously partially differentiable?

Let $f: \mathbb{R^2} \to \mathbb{R}$ with $f(x,y) := 6y+4x-x^3y$ How can one argue that $f$ is infinitely often continuously partially differentiable? I also calculated the gradient $\nabla f: \mathbb{...
user avatar
2 votes
1 answer
88 views

When does the Hessian on a Riemannian manifold vanish?

The Hessian tensor of a smooth function $f:M\rightarrow \mathbb{R}$ on a Riemannian manifold $M$ with respect to the Levi-Civita connection $\nabla$ is given, globally and in local coordinates $\{x^i\}...
user avatar
  • 273
1 vote
1 answer
50 views

Does $\det \mathcal{H}_f = (n-1)(-1)^{2n+1} \left[ f(x_1, \dots, x_n) \right]^{n-2}$ for $f(x_1, \dots, x_n) = \prod_{j=1}^n x_j$?

Let function $f : \mathbb{R}^n \to \mathbb{R}$ be defined by $$f(x_1, \dots, x_n) := \prod_{j=1}^n x_j$$ Like any $C^2$ function, we can compute its Hessian $\mathcal{H}_f$, which will be a $\mathbb{R}...
user avatar
  • 1,328
0 votes
1 answer
37 views

Prove convexity with hessian matrix

I've got a function $$f(x)=(x_1-1)^2+\sum_{i=2}^n (x_i-x_{i-1})^2\quad \text{with $x\in \mathbb{R}^n$}$$ I want to show that this is (strictly) convex, so I thought the best approach might be to look ...
user avatar
  • 1,406
1 vote
0 answers
25 views

If the Hessian is a constant matrix and $\vec{0}=\nabla f(\vec{y})+\nabla ^2 f(\vec{y})\vec{d}.\quad$ Then $\nabla f(\vec{y}+\vec{d})=\vec{0}$

Show that if the Hessian $\nabla ^2 f(\vec{x})\in\mathbb{R}^{nxn}$ is a constant matrix (independent of $\vec{x}$) and a vector $\vec{d}\in\mathbb{R}^n$ is a solution of the system $\vec{0}=\nabla f(\...
user avatar
1 vote
0 answers
61 views

If the Hessian is so Difficult to Calculate - Why do some Optimization Methods still use the Hessian?

I can understand that it will take a computer more time to calculate, approximate and invert the Hessian Matrix (of some function) compared to the Jacobian Matrix - apparently, it is this fact which ...
user avatar
-1 votes
1 answer
23 views

How to find hessian matrix for more than 2-variable function

I know how to find a 2 x 2 Hessian matrix but for machine learning, I'm getting confused since my multivariable calc class only dealt up to three dimensions, and in ML we're working with way more than ...
user avatar
  • 1,264
0 votes
0 answers
39 views

What is a bilinear concomitant (or conjunt) with regards to linear differential operators

Context I am studying self-adjoint eigenfunction problems using [1]. I am working through Example 1 on page 54 in [1]. Example 1 (page 54 in [1]) Suppose we have the linear differential operator $$ L[...
user avatar
4 votes
1 answer
37 views

Extrema of a surface $z=f(x;y)$ when $det(H)=0$

I'm given the following problem: $\text{Examine}\ z=f(x;y)=x^4+y^4+18xy-9x^2-9y^2+1\text{ for extrema and saddle points.}$ It is trivial to find $\nabla f=(4x^3+18y-18x; 4y^3+18x-18y)$ and the ...
user avatar
0 votes
1 answer
69 views

Computing the hessian of a quadratic form

I have a quadratic for that is expressed as: $q = f^{T}Af$ , where $A$ is an $n x n$ symmetric matrix and $f$ a $nx1$ vector output of a sigmoid function which equals $f = \frac{1}{1 + e^{-w^TX}}$ I ...
user avatar
  • 105
1 vote
1 answer
37 views

On Hessians of inverse vector functions

Let have a function $f:R^{n}\longrightarrow R^{n}$ given by $x_{i}\stackrel{f}{\longrightarrow}y_{i}=y_{i}(x_{1},\ldots,x_{n})$, where $f$ is smooth and non-singular in the domain of interest. The ...
user avatar
  • 179
5 votes
1 answer
186 views

Problem 9 M.L Krasnov variational calculus

Warning: Finding extreme value of a multivariable function My question differs from this since I try to use the Hessian criterion so it is not a repeated question. My question: Problem 9 M.L Krasnov. ...
user avatar
0 votes
0 answers
26 views

What's the relation between quadratic growth and convex?

We define quadratic growth as $f(x)-f(x_p)\ge \frac{\mu}{2}\|x-x_p\|^2$, where $x_p$ is the minimizer. I want to know what's the relation between $f$ is quadratic growth and $f$ is convex.
user avatar
  • 2,443
1 vote
1 answer
35 views

Global minimum of multivariable function with monotone increasing partial derivative.

Given a multivariable function $f(x_1,...,x_n)$ and assume that $\frac{\partial f}{\partial x_i}$ are monotone increasing with respect to $x_i$. This means the diagonal of Hessian matrix are positive. ...
user avatar
0 votes
1 answer
43 views

Giving $\nabla^{2} f(\mathbf{x}) \succeq mI$,how to calculate $\|(\nabla^{2} f(\mathbf{x}))^{-1}\| $

In the 《Introduction to Nonlinear Optimization Theory, Algorithms, and Applications with MATLAB》page 85, "Combining the latter equality with the fact that $\nabla^{2} f(\mathbf{x}) \succeq mI$ ...
user avatar
0 votes
0 answers
39 views

what's the concrete formula of the second-order taylor expansions for real-valued functions with complex variables?

I read a paper where the second-order taylor expansions for real-valued functions with complex variables is given as follows: $f(x)=f(x_{0})+\Re\{\triangledown^{H}f(x_{0})\cdot(x-x_{0})\}+\frac{1}{2}(...
user avatar
0 votes
0 answers
35 views

Show the Hessian is locally Lipschitz continuous

The function $f: \mathbb{R}^2 \rightarrow \mathbb{R}$ $f(x,y) = (y-\cos x)^2 + (y-x)^2$ Has the following Hessian $\nabla ^2 f = \begin{pmatrix} 2(\sin^{2}x + (y-\cos x)\cos x + 1) && 2(\...
user avatar
0 votes
0 answers
16 views

Does the order of differentiation matter for the cumulative density function of a multivariate random variable?

consider $F(x,y) = P(X\leq x,Y\leq y)$. Does $\frac{d^2F}{dxdy} = \frac{d^2F}{dydx}$? Also the symmetry means the hessian exists right? What would be the meaning behind such symmetry?
user avatar
  • 519
0 votes
0 answers
34 views

I need a reference for a Lagrange multipliers proof.

I am writing up a proof in which we have to find the maximum of a function $f:\mathbb{R}^n \rightarrow \mathbb{R}$ subject to the constraint $x_1+...+x_n=C$. Naturally, we can use Lagrange multipliers,...
user avatar
  • 1,365
0 votes
0 answers
36 views

Gradient and Hessian for $\|\langle A, X\rangle\|^2$

What is the Gradient and Hessian for $\max(\langle A, X\rangle, 0)^2$, where $A, X$ are both matrices and $\langle \cdot, \cdot \rangle$ is the Frobenius product? Is the gradient just $2\langle A, X\...
user avatar
  • 710
0 votes
0 answers
41 views

Hessian determinant of a function

I have a function with two variables ($f(x,y)$). Hessian determinant of the function is always equal to zero: $f_{xx}f_{yy}-f_{xy}f_{yx}=0 ~~\forall x,y $ and both $f_{xx}$ and $f_{yy}$ are always ...
user avatar
  • 1
2 votes
0 answers
25 views

Proving there is $x\in\Bbb R^n,\|x\|=1$ s. t. $f(x)>f(y),\forall y\in B(0,1),$ where $f\in C^2(A,\Bbb R)$

Let $A\subseteq\Bbb R^n$ be an open set which contains the closed unit ball $\overline{B(0,1)}$ and let $f\in C^2(A,\Bbb R).$ Suppose the Hessian $H_f(c)$ is positive definite $\forall c\in B(0,1).$ ...
user avatar
0 votes
0 answers
39 views

Positive semidefinite matrix with 0 on diagonal

If I have a Hessian matrix $ \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} $ might someone help me understand why this is not positive semidefinite? My understanding that if for ...
user avatar
0 votes
0 answers
24 views

How does Lipschitz Continuity of the gradient implies that the Hessian-Matrix minus Lipschitz-Constant is negativ semidefinit? [duplicate]

I was reading following lecture notes https://www.stat.cmu.edu/~ryantibs/convexopt-F13/scribes/lec6.pdf In the proof the author says that if the gradient $\nabla f$ is Lipschitz-continuous than $H_f(x)...
user avatar
  • 1
1 vote
2 answers
36 views

Differentiating row vector vs column vector when deriving the Hessian

Consider the one-layer neural network $y=\mathbf{w}^T\mathbf{x} +b$ and the optimization objective $J(\mathbf{w}) = \mathbb{E}\left[ \frac12 (1-y\cdot t) \right]$ where $t\in\{-1,1\}$ is the label of ...
user avatar
  • 145
0 votes
2 answers
110 views

Why is this symmetric rank one update "the symmetric solution that is closest?"

Trying to understand symmetric rank one updates and there is this like in the Wikipedia page that says... A twice continuously differentiable function $x\mapsto f(x)$ has a gradient ($\nabla f$) and ...
user avatar
  • 603
1 vote
1 answer
36 views

How to prove that Squared Error Loss is convex

I have the squared loss given by: $L(z;\textbf{y}) = \frac{1}{2}\left\| z-\textbf{y} \right\|^{2}_{2} = \frac{1}{2}(z-\textbf{y})^{T}(z-\textbf{y})$ where $z=\textbf{W}^{T}\phi(x)+\textbf{b}$. I need ...
user avatar
0 votes
1 answer
64 views

Prove the existence and uniqueness of a global minimum for a quadratic function $q(x)$ with positive definite Hessian

How do I prove the existence and uniqueness of a global minimum for a quadratic function $q(x)$ with positive definite Hessian?
user avatar
0 votes
0 answers
51 views

Hessian approximation from Jacobian

I've seen two expressions for using the Jacobian to compute an approximation for the Hessian, and I don't see how they can give you the same answer in general. The papers are here(1) and here(2) (...
user avatar
  • 1
1 vote
0 answers
67 views

Formula for the induced Hessian

I'm reading a paper and I came across a computation that is giving me trouble. Here is the setup. Let $(M,g)$ be a Riemannian manifold and $u:M\to\mathbb{R}$ a smooth function with $\nabla u \neq 0$. ...
user avatar
0 votes
0 answers
37 views

If $H_f(c)$ is positive-definite for each $c$ in the open unit ball, prove $\exists x\in\Bbb R^n,\|x\|=1, f(x)>f(y), \forall y\in B(0,1)$

Let $A\subseteq\Bbb R^n$ be an open set which contains a closed unit ball $\overline{B(0,1)}$. Let $f:A\to\Bbb R$ be a function of the class $C^2$. Suppose the Hessian matrix $H_f(c)$ is positive-...
user avatar
0 votes
0 answers
31 views

Taylor expansion of a sum of vectors

I came across this problem in class where I had calculate the Taylor expansion of $f(\theta + \theta')$ where $f \in C^3(\mathbb{R}^3)$ using the hessian matrix $H_f$. I was told that the solution was ...
user avatar

1
2 3 4 5
15