Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hessian-matrix]

The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both positive and negative eigenvalues then $x$ is a saddle point for $f$.

0
votes
0answers
13 views

How to take derivative of multivariate Taylor series matrices?

Suppose we consider a function $f(x)$ with $f(x):\mathbb{R}^n \to \mathbb{R}$. We let $r(x)$ be the second order Taylor series of $f(x)$ about the base point $z \in \mathbb{R}^n$. How can I show ...
4
votes
2answers
65 views

What is the Hessian of the spectral norm?

The spectral norm of a symmetric matrix is the absolute value of the top eigenvalue. The gradient of this norm is $uu^T$ where $u$ is the eigenvector associated with that top eigenvalue. Assume that $...
2
votes
1answer
27 views

Hessian Matrix 2 variables numerical method

I have a $\mathbb{R}^2\to \mathbb{R}$ function f and I only know 9 points x-h1 x x+h1 y-h2 a d g y b e h y+h2 c f i I ...
0
votes
1answer
21 views

Evaluate extreme point based on eigenvalue & eigenvector of Hess Matrix

Is there a case, in which at an extreme point there are more than 1 positive EValue? This would lead to more than 1 EVector that shows the steepest increase of a function. Up till now I haven't run ...
0
votes
1answer
28 views

Second Partial Derivative Test using Hessian Determinant

I understand that the Hessian determinant (detH) for a function f (x,y) is defined as: \begin{vmatrix} f_{xx} & f_{xy}\\ f_{yx} & f_{yy}\\\end{vmatrix} Where the determinant is a factor for ...
0
votes
1answer
23 views

Characterize the critical points of $\Vert Ax - b \Vert^{2}$

Let $A$ a $m \times n$ matrix, $b$ a $m \times 1$ matrix and $x$ a $n \times 1$ matrix. Consider $f: \mathbb{R}^n \to \mathbb{R}$ defined by $$f(x) = \Vert Ax - b \Vert^2.$$ Determine a condition ...
0
votes
0answers
18 views

Eigenvector of Hess Matrix (Optimisation problem)

I am able to find eigenvalues of Hess Matrix (after plugging in the values of extreme point) but i cant find the eigenvectors of it. The row echleon form of the Matrix shows that it's a full rank ...
1
vote
1answer
92 views

How to show negative entropy function $f(x)=\sum_{i=1}^nx_i\log(x_i)$ is strongly convex?

Let $x \in \mathbb{R}^n$ belongs to $S$ where $$ S= \{x \in \mathbb{R}^n \mid x \succ 0, \|x\|_{\infty} \leq M\} $$ where $\succ$ is the generalized inequality which means all elements of $x$ are ...
0
votes
0answers
31 views

Newton's Method for a step size to move in the direction of the gradient

I am reading this article that talks about Newton's method that can give us an ideal step size to move in the direction of the gradient. I do not understand what $\epsilon$ is in the following part ...
0
votes
0answers
25 views

Taylor expansion on PRML(5.28)

I'm reading PRML and I have a question,don't solve by myself. So, please anyone help me. PRML Q:Why following third time is not multiplied with $E(\boldsymbol{\hat{w}})$ $\cfrac{1}{2} (\boldsymbol{...
0
votes
1answer
36 views

Find the extremas of $f(x,y,z)=xyz-x^2-y^2-z^2$

Find the extremas of $f(x,y,z)=xyz-x^2-y^2-z^2$. After some calculation, $Df(x,y,z)=0$ for and only for $$ \\p_1=(0,0,0), p_2=(2,2,2),p_3=(-2,-2,2),p_4=(-2,2,-2),p_5=(2,-2,-2)\ \\ H(f)=\begin{vmatrix}...
0
votes
0answers
18 views

Trouble calculating the Laplace-Beltrami operator through this formula

Let $U$ be an open, bounded and connected subset of $\mathbb R^3$ with a $C^2−$regular boundary $\partial U$. For an arbitary $x_0 \in \partial U$ define the function $f:B(x_0,r) \cap \partial U \to \...
1
vote
0answers
18 views

How to classify the degenerate stationary points of a multivariate function?

I have a multivariate function in one of whose critical points the Hessian matrix is singular. Is there any general method to determine the type of this critical point? Would be worth plotting the ...
0
votes
1answer
29 views

How to study the critical points of a $2$-variable function?

I am revising some past exam questions and there is one that states: Study the critical points of the function: $$f(x,y)=x^2+y^2-x^4-y^4-2x^2y^2.$$ According to my professor, this is what I have ...
0
votes
0answers
22 views

Hessian matrix is (strictly) negative definite at some point

Give a smooth function $f(x,y)$ on $\mathbb{R}^2$ and let $D = [0,1]\times[0,1]\subset \mathbb{R}^2$. Suppose $f$ is negative on the boundary $\partial D$ but reaches a positive maxima in the interior ...
0
votes
1answer
14 views

Prove $y^tH_f(a)y \leq 0$ with Taylors Theorem

Let the function $f \in C^2(\mathbb{R}^n;\mathbb{R})$ have a local maximum in the point $a \in \mathbb{R^n}$. How can one prove the following with Taylor's theorem: The following applies: $y^tH_f(a)...
3
votes
1answer
63 views

Is there a systematic way to construct functions with prescribed local extrema?

I'm teaching multivariable calculus and having a hard time coming up with optimization problems. Suppose I have three lists of points $\{a_1, \dotsc, a_r\}$, $\{b_1, \dotsc, b_s\}$, and $\{c_1, \...
0
votes
0answers
26 views

Maximum of a two variable function within a defined domain

The function is this: $xye^{\frac{(x+y^2)}{4}}$ in the domain $x+y\geq1, y\geq0$ I have done the partial derivatives to get the stationary point $(0,0)$ but it is not a maximum, it is a saddle point. ...
0
votes
1answer
87 views

Using the Hessian Matrix to classify points

From what I've gathered from my calculus supplements and the web, I want to know if I have the general computation procedure understood correctly. Example: Given f such that f(x,y) = ___. Find and ...
1
vote
0answers
42 views

Approximation of Inverse Hessian or Inverse Hessian Square Root times a vector

I know there are good methods for approximating a Hessian times a vector without actually forming the hessian. (Example here). Are there any methods of approximating the Inverse of hessian times a ...
0
votes
1answer
16 views

Calculating the Hessian of a function $f :\mathbb{R}^{n} \rightarrow \mathbb{R}$ given an expression for the gradient including eigenvalues

We are given a function $f :\mathbb{R}^{n} \rightarrow \mathbb{R}$ such that: $\nabla{f}(x) = \lambda x$ where $\lambda$ is a scalar, and an eigenvalue of some Matrix $A \in M_{n \times n}$ that ...
1
vote
1answer
48 views

Calculating the Hessian of a function $f :\mathbb{R}^{n} \rightarrow \mathbb{R}$ given the gradient

We are given a function $f :\mathbb{R}^{n} \rightarrow \mathbb{R}$ such that: $\nabla{f}(x) = Px$ where $P \in M_{n \times n}$ is an $n \times n$ matrix, and $x \in \mathbb{R}^{n}$. What is the ...
0
votes
1answer
50 views

Matrix calculus : Derivative with respect to two vectors

I have a function of two variables, $\vec{x_i} \in \mathbb{R}^n $ and $\vec{y_j} \in \mathbb{R}^n$: $f(\vec{x_i},\vec{y_j}) =\sum_\limits{i,j} \left( \vec{x_i} \cdot \vec{y_j} + z_{ij} \right)^2$ I ...
1
vote
1answer
74 views

Number of Solutions to the Problem: Minimize $|| A - Afc^T / c^T f ||^2$ such that $\sum_i c_i = 1$

Let $A$ be a $M$ by $N$ matrix, $f$ is a column matrix with $N$ elements, $c$ is a column vector with $N$ elements that I need to solve for, and $|| \cdot ||$ is the Frobenius norm of the matrix. I ...
0
votes
1answer
90 views

Hessian wrt. MATRIX-VARIABLE for a Quadratic Inner Product.

Given standard matrix inner product, \begin{equation} \begin{aligned} f(\textbf{X}) := & \;\;\;\; {\langle}{\textbf{X}, \textbf{A}\textbf{X}}{\rangle}\\ =& \; \text{tr} (\textbf{X}^{\text{...
0
votes
1answer
32 views

At a Critical Point the Hessian Equals the Frst Nonconstant Term in the Taylor Series of $f$?

My textbook defines Hessian functions as follows: Suppose that $f: U \subset \mathbb{R}^n \to \mathbb{R}$ has second-order continuous derivatives $\dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\...
0
votes
0answers
20 views

hessian calculation with division by zero for first derivative

I am wanting to calculate the hessian from an example dataset that keeps evolving. At one particular data set, one of the parameters (z) does not change, which causes a division by zero for the ...
0
votes
0answers
29 views

Hessian matrix and analysis

I've been trying to solve this problem for a long time, but I don't know exactly how to start: Translation: M is square dxd, we want to show that f belongs to C^2 in R^d. After we want to calculate ...
0
votes
1answer
194 views

Hessian of negative log-likelihood of logistic regression is positive definite?

I'm trying to show that the Hessian of the negative of the log likelihood with two parameters is positive definite, but I'm not sure how to go about it once I compute the Hessian. The function is: $-...
0
votes
0answers
19 views

Hessian Matrix Denominator

Isn't the denominator for each diagonal term in the Hessian matrix as given on Wikipedia incorrect? Hessian Matrix Instead of the denominator being $\partial x_1^2 $, shouldn't it be $\partial^2 x_1 ...
0
votes
1answer
67 views

Is $z = x^2y^3(1-x-y)$ convex or concave?

Is there some kind of trick to defining the domain of the concavity/convexity (if it exists)? I have no idea how to work with the resultant hessian
2
votes
1answer
76 views

gradient and hessian of $e^{x^Tx}$

I want to find gradient and hessian of $e^{x^Tx}$ My attempt: $\nabla = 2x^Te^{x^Tx}$ Hessian $= 2e^{x^Tx}I + 2xx^Te^{x^Tx}$ Is that correct?
0
votes
0answers
32 views

Regular surface with Curvature mean constant

Let $F:R^3->R$ given by $F(x,y,z)=ax^2+by^2+cz^2 - 1$ I wanto to proof that inverse image of F apply at point $0$ is a regular surface with curvature mean constant. Let me tell you what my first ...
0
votes
0answers
19 views

Interpretation of signs and magnitudes of eigenvalues of Hessian

Suppose I have a 50-dimensional field. I compute the Hessian matrix at a stationary point and find 40 negative eigenvalues + 10 positive ones. Can I conclude that the point is "mostly" a maximum with ...
0
votes
1answer
36 views

Sufficient Conditions for quasiconcavity proof

I was reading a book and it says that the sufficient condition for a function to be quasiconcave is that its Bordered Hessian matrix is negative definite. I can't seem to understand this. Please help! ...
0
votes
1answer
28 views

Hessian and gradient in a matrix

There is following matrix: $$\begin{pmatrix}\nabla^2g(x) & \nabla g(x)\\\ \nabla g(x)^{T} & 1\end{pmatrix} \ge 0$$ the "$\ge$" is general inequality (not element-wise), meaning the matrix ...
0
votes
1answer
55 views

Do you agree that the following domain is not convex?

The domain is given as $x_1,x_2,y_1,y_2 \in \mathbb{R}$ with: $$x_1-y_1^2-4 \geq0,\quad x_2-y_2^2-4 \geq 0, \quad x_1\leq 10 ,\quad x_2 \leq 10 $$ We must prove this is convex. This is my approach: I ...
2
votes
0answers
60 views

A question regarding convex function and Hessian

I came across with the problem which asks me to show that: $f:\mathbb{R}^n\rightarrow \mathbb{R}, f\in C^2$ with Hessian of $f$ well-defined on $\mathbb{R}^n$, is convex if and only if $\nabla^2f\...
0
votes
1answer
49 views

Convexity and concavity with Hessian matrices

I want to analyze two Hessian matrices regarding definiteness to formulate conclusions whether the functions are convex or concave. If you could check my thoughts, I’d be grateful. $H_1(x)=\left ( \...
0
votes
1answer
22 views

Find and classify the stationary points for $f(x,y) = (4x_1^2 - x_2)^2$

So first I calculated the gradient which was $\nabla f(x) = (64x_1^3 - 16x_1x_2, -8x_1^2 + 2x_2)^T.$ Then setting the equations in this equal to $0$ I solved for $x_2$ and got $x_2=4x_1^2$. Then ...
0
votes
2answers
75 views

Clarification of Textbook Explanation of Hessian Matrix, Directional Second Derivative, and Eigenvalues/Eigenvectors

My machine learning textbook has the following section on the Hessian matrix: When our function has multiple input dimensions, there are many second derivatives. These derivatives can be collected ...
2
votes
1answer
70 views

Hessian of a $C^{\infty}$ function $f: \mathbb R^n \to \mathbb R$ restricted to a subspace

Let $f: \mathbb R^n \to \mathbb R$ be an infinitely differentiable function. Let $\mathcal V \subset \mathbb R^n$ be a vector subspace. Let $g = f|_{\mathcal V}$ be the restriction of $f$. Then we can ...
0
votes
0answers
38 views

Hessian decomposition?

I came across the decomposition below but I am not sure how to derive it or if it is in fact true. Can anyone share some insight? $D^2v=\partial^2_{nn}v(n^T)+\partial^2_{nr}v (nr^T+rn^T)+\partial^2_{...
0
votes
1answer
37 views

what if the hessian of the Lagrangian is null?

In a non-linear optimization problem, suppose given a candidate solution $x*$, we want to verify second order sufficient conditions of optimality but the Hessian of the Lagrangian of the problem at $x*...
4
votes
0answers
37 views

A matrix problem about egienvalue and trace

Consider an $m\times m$ positive definite and Hermitian matrix $\mathbf{M}$ and an arbitrary $m\times n (m>n)$ para-unitary matrix $\mathbf{R}$, i.e., $\mathbf{R}^H\mathbf{R}=\mathbf{I}_n$. ...
0
votes
2answers
37 views

low-complexity matrix inversion algorithm for a near-identity matrix?

As I know, the general complexity of matrix inversion is $O(n^3)$, but it is a little bit high. My matrix is $(I + A)$ , where $I$ is an $n \times n$ identity matrix and $A$ is a hermitian matrix ...
4
votes
3answers
177 views

Gradient and Hessian of $x x^T$ w.r.t. $x$, where $x \in \mathbb{R}^{n \times 1}$,?

Question: Can we find the gradient and Hessian of $x x^T$ w.r.t. $x$, where $x \in \mathbb{R}^{n \times 1}$ ? EDIT: If we can, may I know how to compute that? Thank you.
1
vote
0answers
59 views

Condition number of Hessian matrix when Hessian is singular

In gradient descent on a quadratic problem, $$\min _{x \in \mathbb{R}^n}\frac12 \langle Hx, x \rangle + \langle b, x \rangle + c \qquad H \text{ symmetric, positive semi-definite}\quad b \in \...
6
votes
2answers
189 views

Calculate the Hessian of a Vector Function

I'm working with optimisation. I am trying to obtain the hessian of a vector function: $$ \mathbf{F(X) = 0} \quad \text{or} \quad \begin{cases} f_1(x_1,x_2,\dotsc,x_n) = 0,\\ f_2(x_1,x_2,\...
0
votes
2answers
91 views

Example of convex function which is differentiable, but not twice differentiable?

Are there convex functions for which hessian is not defined, but the gradient is defined everywhere? I was looking at projected gradient descent, as well as Newton's method for solving optimization ...