Questions tagged [hessian-matrix]
The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both positive and negative eigenvalues then $x$ is a saddle point for $f$.
826
questions
0
votes
0
answers
23
views
Eigenvalue Optimization, Find indefinite matrix with only one negative eigenvalue
I am new to the field of eigenvalue optimization. Say I have a symmetric matrices $A(x)\in\mathbb{R}^{2n\times2n}$ which depend on $x\in\mathbb{R}^n$. I now want to find situations in which one and ...
- 11
0
votes
1
answer
36
views
Is this multivariate function convex? And is a Hessian matrix of this form always convex?
I'm looking at the following function:
$$K(Q, s) = h\frac{(Q-s)^2}{2Q}+b\frac{s^2}{2Q}+K\frac{\lambda}{Q}+c\lambda,$$
where $Q>0$, $s \ge 0$ and $h,b,c, \lambda, K \ge 0$ and are known.
I want to ...
- 1,407
4
votes
1
answer
165
views
Is Hessian with zero directions indefinite?
I have a Hessian matrix $\mathbf{H}$ of a function $f(\mathbf{x})$, evaluated at an extreme point $\mathbf{x}_0$. Lets assume $\mathbf{H}$ is non-singular.
I can show that there exists a direction $\...
- 285
0
votes
0
answers
45
views
Symmetric bilinear form from Hessian
Assuming a PDE is given by the Hesse-operator
$$
-\nabla \otimes \nabla \lambda = \mathbf{F} \, ,
$$
then testing the left-hand-side with a symmetric tensor $\mathbf{S}$ leads to
$$
-\langle \mathbf{...
- 111
-1
votes
0
answers
24
views
Determinant of hessian matrix
Assume that $f:\mathbb{R}^n\to\mathbb{R}$ is smooth function and determinant of its hessian matrix always $2$. By inverse function theorem, $F(x)=\nabla_x f(x) $ ($x\in\mathbb{R}^n$) has inverse ...
- 1
0
votes
0
answers
51
views
Hessian of the Rayleigh quotient $\frac{\langle x,Ax\rangle}{\langle x,x\rangle}$
I am struggling on the following question. Let $A$ be a semidefinite positive matrix ($A\in\mathscr{S}_n^+(\mathbb{R})$) and let consider the Rayleigh quotient defined as:
$$\forall x\in\mathbb{R}^n\...
- 305
0
votes
0
answers
70
views
Gradient and Hessian using chain / product rule
I am reading a paper (How Hessian structure explains mysteries in sharpness regularization), and not sure how Eq (2), Pg 2 was derived from Eq (1), using standard product or chain rule.
In general, my ...
0
votes
1
answer
47
views
Gradient & Hessian for log sigmoid function
I have a log sigmoid loss function,
$$
l(\textbf w) = \frac{-1}{n}\sum{\log(\sigma(y_i\textbf w^T\textbf x_i))}
$$
where $y_i$ is the class label which could be 1 or -1, $\textbf w^T$ is the vector ...
- 9
0
votes
0
answers
26
views
Expectation of matrix
Denote a matrix A such that A$\in \mathbb{C}^{M\times N}$, and each element of this matrix is modeled to follow a complex Gaussian distribution with zero mean and unit variance, and each element is ...
- 31
1
vote
0
answers
20
views
Reference for the bordered Hessian method
I need a reference for the proof of the bordered Hessian test. I know how to use it, but I kinda learned it "in the wild" and need a standard good reference for its proof to add to my ...
- 157
0
votes
1
answer
25
views
Hessian matrix of multiplication
Introduction
Lets assume $A(\vec{x})$, $B(\vec{x})$ and $C(\vec{x})$ are functions with n inputs and 1 (the $\vec{x}$ is ...
- 173
2
votes
0
answers
35
views
Uncertainty analysis in maximum likelihood estimation under constraint
I'm not from a statistical background so you might have to excuse me for my somewhat inaccurate (or even erroneous) phrasing, I'll try the phrase my problem as I understand it.
The maximum likelihood ...
- 31
2
votes
3
answers
165
views
Unique solution of linear system involving hessian and gradient
Consider the following problem:
$$
f(x) = \frac{1}{2}x^T Hx + b^T x.
$$
In this case, ($f''(x) = H$) with $H \in \mathbb{R^{n \times n}}$, and hence:
$$
f''(x^{k+1})(x^{k+1} - x^k) = H (x^{k+1} - x^k) ...
- 389
0
votes
0
answers
27
views
Show that $f(x+\alpha h) - f(x) = 1/2(H(x)\alpha h, \alpha h) + o(\alpha^2), h \in \mathbb{R}^n, \alpha > 0$ where $H(x)$ is the hessian of $f(x)$
Show that $f(x+\alpha h) - f(x) = 1/2(H(x)\alpha h, \alpha h) + o(\alpha^2), h \in \mathbb{R}^n, \alpha > 0$ where $H(x)$ is the hessian of $f(x)$. Edit: also $\nabla f(x) = 0$. I can show that $f(...
- 641
1
vote
1
answer
78
views
Does subtracting a rank-1 matrix from a positive definite matrix result in positive semi-definite matrix? [closed]
Let $A$ be a symmetric positive definite matrix. We define the matrix $B$ as follows:
\begin{equation}
B = A - \frac{1}{e^\top A e} Aee^\top A
\end{equation}
where $e=(1,1,...,1)$.
I believe that $B$...
- 29
0
votes
0
answers
26
views
Hessian matrix, Convexity and Concavity
I have a function and would like to get its convexity or concavity property. Different from normal cross-entropy, here $y_i$ is not one-hot, hence the property is bit different with that perfect case.
...
- 1
0
votes
0
answers
26
views
Hessian Matrix calculation for the Harmonic potential function
I would like to find out the Hessian matrix of the following harmonic potential function
$$V=\frac{k}{2}|\vec{r}_i-\vec{r}_j|^2$$ where $r_{ij}^2=|\vec{r}_i-\vec{r}_j|^2=(x_i-x_j)^2+(y_i-y_j)^2+(z_i-...
- 182
2
votes
1
answer
93
views
Computation of Riemannian Hessian of the Stiefel manifold
I am fairly new to differential geometry, so I'm sorry if I am sloppy in my notations.
In the book Optimization Algorithms on Matrix Manifolds, the Riemannian Hessian is given by definition 5.1 as:
$$...
2
votes
0
answers
121
views
Intuition for the inverse Hessian matrix
Consider a function $f:\mathbb{R}^n\to\mathbb{R}$ and denote by $H$ its Hessian matrix. I understand that $H$ provides a measure of the curvature of $f$ in all directions and plays a role in the ...
- 1,335
2
votes
0
answers
39
views
Hessian matrix = zero matrix? [closed]
Is there something we can tell about the function, if we know that its Hessian matrix is the zero matrix? Is it concave, convex? What about quasi-concave and quasi-convex? As I am a beginner on this ...
0
votes
0
answers
18
views
Finding Local Extrema of Multi-Variable Function
I'm trying to find the local extrema of a multi-variable function...kinda lost as to whether or not my approach is correct. If someone can look this over and tell me if i'm wrong, and if so, whats the ...
0
votes
2
answers
58
views
Minimize a quadratic function of three variables over a plane
I have the following minimization problem:
If $ x + 2 y + 4 z = 10 $, what is the minimum of
$ f(x,y,z) = x^2 + 2 y^2 + 3 z^2 + x y + 2 xz + 3 y z + 7 z $
and at what value of $(x,y,z)$ does the ...
- 19.1k
0
votes
0
answers
37
views
Application of multidimensional Taylor's Formula
I need to show an inequality that I heavily suspect to involve Taylors's formula, but I am failing at proving it.
We are given a function $S(x,\theta)$, where $x\in R^{d_1}$ and $\theta\in\Theta\...
- 23
0
votes
0
answers
52
views
Hessian: $\dfrac{\partial^2f(\mathbf{x})}{\partial\mathbf{x}\partial\mathbf{x}^\top}$ or $\dfrac{\partial^2f(\mathbf{x})}{\partial\mathbf{x}^2}$?
I come across two definitions for the Hessian matrix
$$
\begin{align}
\mathbf{H} = \dfrac{\partial^2f(\mathbf{x})}{\partial\mathbf{x}\partial\mathbf{x}^\top} \tag{1}
\end{align}
$$
and
$$
\begin{align}...
- 313
3
votes
0
answers
63
views
Can we classify all functions whose gradient is an eigenvector of the Hessian?
Let's treat the case of two dimensions, then we don't have freedom with the other eigenvector. Is there any classification of all smooth functions $f$ on $\mathbb{R^2}$ such that $\nabla f$ is an ...
-1
votes
1
answer
27
views
k-Jet of a function $f:R^3 \rightarrow R$
I am studying this article. Can someone explain to me why the 4-jet of the function
$U(x, y, z) = x^4 + y^4 - z^6$
is equal to
$(j^{(4)} U)(x, y, z) = x^4 + y^4$
Also, the author says that the origin ...
- 155
1
vote
1
answer
86
views
Meaning of $\alpha I \preceq \nabla ^2 F$
I learned that the notation $\alpha I \preceq \nabla ^2 F $ means that $\langle \alpha x, x\rangle \leq \langle \nabla^2 F x, x\rangle$.
What is this called? I know that $\nabla^2 F \succeq 0$ is ...
- 1,836
0
votes
1
answer
36
views
What is a convex set $S = \mathbb{R}_{++}^2$
I worked on the following task:
The function $f(x) = f(x_1, x_2) = x_1^{1/2} \cdot x_2^{1/2}$ is concave on the convex set $S = \mathbb{R}_{++}^2$.
I could solve the task with the Hessian matrix (...
0
votes
1
answer
31
views
Strict Concavity of a two-variable function
I have the following two-variable function $h(t,w):= t\,\log\left(\dfrac{w}{t}\right), \, t\in\mathbb{R}^{>0},\, w\in\mathbb{R}^{>0} $ and I was trying to prove whether this function is strictly ...
- 183
0
votes
0
answers
109
views
Finding gradient and hessian matrix
I am trying to find the gradient and hessian of a matrix-t-distribution representing a generalized linear dynamic linear model. At each time step k, we get a marginal t-distribution whose log is given ...
0
votes
0
answers
24
views
Non-linear optimization with Equality constraints: Shouldn't a different hessian be defined?
Background
I am studying some theory for non-linear optimization, and i am currently studying about Lagrange multipliers.
According to this, the classical approach to solve an "Equality ...
1
vote
1
answer
84
views
How to compute the gradient and hessian of a vector valued function
I'm trying to implement the algorithm below in matlab but i dont know how to compute the gradient and hessian of a vector valued function, i need to know how to do it to apply what is in the red ...
- 121
0
votes
1
answer
84
views
Can I invert the hessian using row operations like this? [closed]
I put my derivations in this image here: [![enter image description here][1]][1]
I am just using gaussian elimination by integrating all row elements. Is this acceptable?
With this approach, we only ...
- 3,295
1
vote
1
answer
52
views
How do we define the Hessian of a function $f$ of a matrix $X$?
Suppose $f$ is a real-valued function of the matrix $X \in \mathbb{R}^{m \times n}$. We typically define the gradient of $f$, written $\nabla f(X)$, as the $m \times n$ matrix whose $(i,j)$th entry is ...
- 1,337
0
votes
0
answers
49
views
Can I invert a hessian matrix using gauss jordan elimination?
I don't understand why you can't invert a hessian matrix using gauss jordan method. Can't you integrate or differentiate an entire row (because they are linear operators) and then subtract/add/swap?
...
- 3,295
6
votes
2
answers
151
views
What is the Hessian of $x \mapsto\log \det \left( A^T A + R^T \operatorname{diag}(x)^{-1} R \right)$?
This is a follow-up to a previous question I asked regarding the hessian of a similar log determinant. The log determinant I am considering is given by
$$
L(\vec{x}) = \log \det \left( A^T A + R^T D_x^...
2
votes
1
answer
132
views
Hessian matrix of $\Lambda \mapsto y' (I + X\Lambda X')^{-1}y$
I have $$f=y'M^{-1}y$$ where $$M = I + X\Lambda X'$$ for $y \in \mathbb{R}^n$, $X\in \mathbb{R}^{n\times p}$, and $\Lambda$ is a $p\times p$ symmetric positive-definite matrix. I'm trying to compute ...
- 993
0
votes
0
answers
43
views
2nd order EKF covariance propagation equation (hessian)
I am trying to implement a 2nd-order EKF and am having some issues with the propagation equation for covariance.
From the literature, if $X$ has covariance $P$ and the propagation function $f$ has ...
- 261
0
votes
1
answer
40
views
What is the definiteness of this matrix?
I am trying to calculate the extrema of the function $$f(x,y) = \sin(x)\sin(y)\sin(x+y)$$ with the constraint $0 < x, y,x+y < \pi$. I have determined the critical point to be $\left( \frac{\pi}{...
- 31
0
votes
0
answers
80
views
Jacobian and Hessian of $f(x) = \langle x, Ax \rangle$
A is a $\Re^{n \times n}$ Matrix.
f is a function from $\Re^n$ to $\Re$ with $f(x) = \langle x, Ax \rangle$.
How can I determine the gradient and hessian of this Matrix at point x?
user1159114
2
votes
3
answers
192
views
Second order derivative of $f(x):=\frac{1}{2} ⟨x,Ax⟩$
Let $A=\left(A_{i j}\right)$ be an $n \times n$ symmetric matrix, and define the function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ as
$$
f(x):=
\frac{1}{2}
⟨x,Ax⟩
$$
Using the definition, ...
- 101
0
votes
0
answers
27
views
Does a symmetric matrix with only three types of values have always three types of eigenvalues?
The title is quite explicit, however I will give the context. I'm referring to spin-glass theory in classical statistical mechanics, in particular to the Sherrington-Kirkpatrick model. Studying the ...
0
votes
0
answers
20
views
If the Jacobian of a vector field is symmetric, is the vector field a gradient?
Suppose I have a vector field $v$, and I am interested in knowing whether it is a potential gradient, i.e. $v = \nabla f$ for some scalar function $f$. Symmetry of the Jacobian $Dv$ is clearly ...
- 298
0
votes
0
answers
105
views
Critical points with Hessian determinant equals to zero
Consider the function $$f(x, y) = x^4+y^4-y^2$$
After having compute the gradient, I found the following critical points
$$p_0 = (0, 0) \qquad \qquad p_{1, 2} = \left(0, \pm \sqrt{\frac{1}{2}}\right)$$...
- 2,696
1
vote
2
answers
53
views
Positive / negative (and so on) definite matrix. Confusion about the terms
I have problems in understanding the terminology used in Sylvester's Criterion about the "sign" of a matrix.
I got the "positive-definite", the "negative-definite", the &...
- 2,696
0
votes
2
answers
77
views
Local convexity of a function containing norms
Given the function:
$$f(\mathbf{x}, \mathbf{C}) = \prod_{i=1}^N\left(1+\frac{1}{2}\left|x_{i}-0.5\right| + \frac{1}{2}\left|C_{i}-0.5\right|-\frac{1}{2}\left|x_{i}-C_{i}\right|\right) $$
for $\mathbf{...
- 474
0
votes
1
answer
112
views
Second derivative of $\ell_p$ norm of a vector
The $\ell_p$-norm of a vector $\mathbf{x}$ is defined as the root of the sum of the absolute values of its elements raised to the power of $p$.
I want to calculate the following derivative.
$$\frac{\...
- 11
2
votes
0
answers
62
views
Find the local extemum of $x^{2}-2y^{2} - \ln (x + y)$
I have to find the local extrmum of the function $$f(x, y) = x^{2}-2y^{2} - \ln (x + y).$$
I found the first derivatives of $$f'_x = 2x-\frac{1}{x+y}$$ $$f'_y = -4y-\frac{1}{x+y}$$
I made the system $...
- 417
1
vote
2
answers
231
views
How can I prove the Hessian of the log likelihood of the Generalized Error Distribution is negative definite?
I'm working with the multivariate generalized error distribution to model some data. (The parameterization that I am working with follows Graham Giller's work here: https://www.researchgate.net/...
- 11
0
votes
1
answer
49
views
How to prove $\left\Vert\nabla \frac{f}{||\nabla f||+\epsilon}\right\Vert\leq {\rm constant}$?
How to prove $\left\Vert\nabla \frac{f}{||\nabla f||+\epsilon}\right\Vert\leq {\rm constant}$?
The $f:\mathbb{R}^n\rightarrow\mathbb{R}$ is a continuous function. The $\epsilon$ is a small positive ...
- 1