Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hessian-matrix]

The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both positive and negative eigenvalues then $x$ is a saddle point for $f$.

0
votes
1answer
15 views

How do we know that the hessian will be positive definite for MLE of logistic regression parameters?

How does $0<y_n<1$ guarantee that the hessian will be positive definite? $\Phi^T $ is mxn. then $\Phi^TR\Phi$ is mxm. But This doesn't guarantee positive definite. I suppose since R is a ...
2
votes
0answers
58 views

What is the second-order Taylor expansion of a function $f : \mathbb C^n \to \mathbb R$?

Consider for example $f(x)=\|Ax-b\|_2^2$, where $A \in \mathbb C^{m \times n}$, $x \in \mathbb C^n$, $b \in \mathbb C^m$. $y := Ax - b \implies dy = A dx \implies dy^* = A^* dx^*$. Taking the ...
0
votes
0answers
20 views

Hessian of quadratic forms

can you give me a proof that the Hessian matrix of a quadratic form woth associated symmetrix matrix A is equal to 2A? I do not understand all the other proofs I have found.
0
votes
0answers
10 views

Is a diagonally dominant tridiagonal matrix with negative entries in main diagonal negative definite?

If H is a Hessian matrix that is symmetric, diagonally dominant and tridiagonal with negative entries in main diagonal and positive entries in super-diagonals and sub-diagonals. What are the ...
0
votes
1answer
17 views

Maximizing a function with dependent variables

I am trying to find the maximum of a function $f(x,y)$ in which $x$ and $y$ are dependent on each other. For example, $x$ is the size and $y$ is the weight of a component. In order to find the ...
0
votes
1answer
24 views

Hessian Matrix: Meaning is definite

Why does the Hessian matrix being positive definite mean it is a minimum point, and negative definite mean it is a maximum point?
2
votes
2answers
45 views

Classifying the stationary points of $f(x, y) = 4xy-x^4-y^4 $

$f(x, y) = 4xy-x^4-y^4 $ The gradient of this function is $0$ in $(-1, -1), (0, 0),(1, 1)$ I tried to compute the determinant of the Hessian Matrix, but it's 0 for every point, I always get a null ...
0
votes
1answer
24 views

Multivariable calculus - Trying to understand if a stationary point is a saddle point, max or min

I have the following function $f(x, y) = x^4+y^4 $ $(0, 0)$ is a stationary point, so I calculat the determinant of the Hessian Matrix, which is 0, so I try to understand what kind of point that is ...
0
votes
0answers
9 views

Usage of Hessian Matrix to determine extremums of multivariable functions

When I first studied the idea of extremums in multivariable functions and how to be determined I started with 2 variable functions. The idea of Hessian Matrix was introduced to use the sign of its ...
0
votes
1answer
36 views

Proving that a function is convex $\Leftrightarrow f''(x)\geq0$

I am able to prove $\Rightarrow$, but I am unable to prove the converse. Recall that a function is called convex on $E$ (which is a subset of a linear space) if for every $x,y\in E$ and every $\...
2
votes
1answer
37 views

Stationary point of $(x_2 - x_1^2)^2 + x_1^5 $not local max/min

I need to prove that $f(x)=(x_2 - x_1^2)^2 + x_1^5 $ has one stationary point which is neither a local max nor min The stationary point of $f(x)$ is found by $\nabla f(x)=0$ which gives $x_1 = x_2 = ...
0
votes
0answers
22 views

Step size in gradient descent

I am trying to minimize the function $f(X) = \|A - XYY^TX^T\|_F^2$ where the gradient of $f$ follows the bound given belew where $A,X,Y \in R^{n \times n}$ $$\|\nabla f(X_1) - \nabla f(X_2)\|\leq L\|...
0
votes
1answer
30 views

When the Hessian matrix is indefinite, why does the point have to be a saddle point?

Simply a question that occurred to me and proof which I can't seem to find. I realise that if the Hessian matrix is indefinite, it's determinant is less that zero but how does that mean that the point ...
0
votes
0answers
14 views

How to compute the hessian matrix of a class of matrix trace function?

I want to compute the Hessian matrix of a calss of matrix trace function $$f(X)=c+ \operatorname{tr}(AX)+\sum\limits_{j=1}^{m}\operatorname{tr}(Bj X CjX^T).$$ Here $A$ is a fixed $p{\times} n$ matrix;...
0
votes
0answers
17 views

$\mathcal{R}\{\bullet\}$ for matrix-vector product

I am studying about Hessian free optimization methods and I came across a method to convert Hessian into Gauss Newton approximation: $H \approx J^THJ = G$, here G is the Gauss newton approximation. ...
0
votes
1answer
43 views

Gradient of $g(x) = f(Ax + b)$

I need the gradient and Hessian of the function $g(x) = f(Ax + b)$. $f:\!R^m \rightarrow \!R$, $x \in \!R^n$, $b \in \!R^m$, $A \in \!R^{mxn}$ I cannot find the expression for the ...
2
votes
1answer
59 views

Example of “The eigenvalues of data covariance matrix, $\Phi^T\Phi$ measure the curvature of the likelihood function.”

I am reading PRML, Chapter 3.5.3, screen shot attached. I can understand the derivation and maths but hard to understand the meaning of "The eigenvalues of data co-variance, $\Phi^T\Phi$ matrix ...
0
votes
1answer
43 views

Hessian of quadratic form of function using Hadamard and Frobenius notation

Related to this question, I am trying to compute the Hessian of $$ g(r, \theta) = [r\cos(\theta)]^{\top} A \, [r\cos(\theta)] = f(r, \theta) ^{\top} A \, f(r, \theta) \tag{$*$} $$ for $r, \theta \in \...
0
votes
0answers
34 views

Positive semi-definiteness?

One of my homework solutions stated the following: However, I do not seem to grasp why the solutions refers to 'both principal minors'. As far as I know there are three principal minors: Order 1: ...
0
votes
0answers
16 views

Does the differential of a flow fix the kernel of Hessian at critical point?

While learning about degeneracy of critical points in the context of Morse theory, I formulated the following question. Possibly it is a simple consequence of the so-called "first variation" equation ...
3
votes
0answers
25 views

$\mathbb{C}\mathbb{R}$-calculus for quadratic form

I've been working through this set of notes on differentials of $\mathbb{R}$-valued functions of complex variables (and this MSE question), and I'm trying to work through a simple example. Do I have ...
0
votes
1answer
36 views

Find a descent direction at a saddle point

question: (Descent directions at stationary points). Assume that $f: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is $C^2$ with $\nabla f(x) = 0$ and $\nabla^2f(x)$ indefinite (with both positive and ...
0
votes
0answers
34 views

I am trying to find the maximum learning rate or stepping rate of steepest descent algorithm in 2 dimensions

Let $f(x,y) = (x-y)^4+2x^2+y^2-x+2y$. I am trying to numerically find the miniumum of $f$. We define a fixed-point iteration scheme \begin{align*} g(x, y) = \vec{x}_{n+1} = \vec{x}_{n} - a\nabla f \...
1
vote
1answer
56 views

Prove that function has minimum when Hessian is zero

Prove that the following function has a minimum at $(0,0)$. $$ f(x,y) = \dfrac{1}{24}[e^{2y+2\sqrt{3} x}+e^{2y-2\sqrt{3} x} + e^{-4y}] - \dfrac{1}{8} $$ My attempt: I tried to solve it via ...
0
votes
0answers
21 views

Need help calculating partial derivatives of radial function

I'm really stumped (lost more like) on exactly what the Hessian matrix is composed of, how do I calculate the partial derivatives inside the matrix? Am I using $u(x,y)=\sqrt{x^2+y^2}$? Or am I dealing ...
1
vote
1answer
62 views

Bordered Hessian matrix to find a minimum of the function

I was trying to find the global minimum for the function $$(a + b) z + (a + c) y + (b + c) x $$ subject to the following constraint: $$(xy + xz + yz)(ab + bc + ac)=1.$$ By Lagrange multipliers I found ...
0
votes
0answers
74 views

Evaluation of the integral using the Laplace approximation?

How can I evaluate the following integral using the Laplace approximation at any given point $x$? \begin{equation} x \mapsto \int \sigma(w^T x) \mathcal{N}(w; 0, \Sigma) dw \,, \end{equation} ...
2
votes
2answers
58 views

Minimum of $x_1^2+x_2^4$

I am asked to find a minimum of $f(x_1, x_2) = x_1^2+x_2^4$ applying the optimality conditions. I am stuck. What I have found is not conclusive. I show what I have done: Testing the First-Order ...
0
votes
1answer
31 views

Critical points of multi-variable function.

I have the expression: $(x^3-y^2 )(x-1)$ and have to find the critical points and their nature. The points found are: $(1,1) = $ saddle point $(1,-1) = $ saddle point $(\frac 3 4 ,0) = $ global ...
0
votes
0answers
18 views

How to get the partial information matrix from the covariance matrix

I have four random variables${\left( {x,y,z,\varphi} \right)^T}$. And I know its covariance matrix $Cov=\left[ {\begin{array}{*{20}{c}} {\sigma _{xx}^2}&{\sigma _{yx}^2}&{\sigma _{zx}^2}&{...
0
votes
0answers
16 views

Minimal eigenvalue of hessian matrices

I've got a problem that seems to be wrong to me. Some clarification would be of great help! Let $f:\mathbb{R}^n \to \mathbb{R}$ be twice continuously differentiable with local minimum $\overline{x}$ ...
0
votes
0answers
30 views

What is the Hessian of $(\min(0, A) ) ^ 2$

Suppose I have a matrix $A \in R^{m \times n}$. I would like to know the Hessian of $\min(0, A)^2$. I have computed the gradient which is $2 \min(0,A)$ (am I right ?) but I am stuck in finding the ...
2
votes
0answers
61 views

Why is the second derivative denoted by $\frac{\partial}{\partial x \partial x^T}$ in matrix calculus?

When trying to derive RSS of linear model using denominator layout $\frac{\partial}{\partial\beta}(y - X\beta)^T(y - X\beta) = \frac{\partial(y -X\beta)}{\partial\beta}\frac{(y - X\beta)^T(y - X\...
0
votes
0answers
15 views

How to take derivative of multivariate Taylor series matrices?

Suppose we consider a function $f(x)$ with $f(x):\mathbb{R}^n \to \mathbb{R}$. We let $r(x)$ be the second order Taylor series of $f(x)$ about the base point $z \in \mathbb{R}^n$. How can I show ...
5
votes
2answers
85 views

What is the Hessian of the spectral norm?

The spectral norm of a symmetric matrix is the absolute value of the top eigenvalue. The gradient of this norm is $uu^T$ where $u$ is the eigenvector associated with that top eigenvalue. Assume that $...
2
votes
1answer
45 views

Hessian Matrix 2 variables numerical method

I have a $\mathbb{R}^2\to \mathbb{R}$ function f and I only know 9 points x-h1 x x+h1 y-h2 a d g y b e h y+h2 c f i I ...
0
votes
1answer
24 views

Evaluate extreme point based on eigenvalue & eigenvector of Hess Matrix

Is there a case, in which at an extreme point there are more than 1 positive EValue? This would lead to more than 1 EVector that shows the steepest increase of a function. Up till now I haven't run ...
0
votes
1answer
36 views

Second Partial Derivative Test using Hessian Determinant

I understand that the Hessian determinant (detH) for a function f (x,y) is defined as: \begin{vmatrix} f_{xx} & f_{xy}\\ f_{yx} & f_{yy}\\\end{vmatrix} Where the determinant is a factor for ...
0
votes
1answer
27 views

Characterize the critical points of $\Vert Ax - b \Vert^{2}$

Let $A$ a $m \times n$ matrix, $b$ a $m \times 1$ matrix and $x$ a $n \times 1$ matrix. Consider $f: \mathbb{R}^n \to \mathbb{R}$ defined by $$f(x) = \Vert Ax - b \Vert^2.$$ Determine a condition ...
0
votes
0answers
23 views

Eigenvector of Hess Matrix (Optimisation problem)

I am able to find eigenvalues of Hess Matrix (after plugging in the values of extreme point) but i cant find the eigenvectors of it. The row echleon form of the Matrix shows that it's a full rank ...
1
vote
1answer
135 views

How to show negative entropy function $f(x)=\sum_{i=1}^nx_i\log(x_i)$ is strongly convex?

Let $x \in \mathbb{R}^n$ belongs to $S$ where $$ S= \{x \in \mathbb{R}^n \mid x \succ 0, \|x\|_{\infty} \leq M\} $$ where $\succ$ is the generalized inequality which means all elements of $x$ are ...
0
votes
0answers
40 views

Newton's Method for a step size to move in the direction of the gradient

I am reading this article that talks about Newton's method that can give us an ideal step size to move in the direction of the gradient. I do not understand what $\epsilon$ is in the following part ...
0
votes
0answers
25 views

Taylor expansion on PRML(5.28)

I'm reading PRML and I have a question,don't solve by myself. So, please anyone help me. PRML Q:Why following third time is not multiplied with $E(\boldsymbol{\hat{w}})$ $\cfrac{1}{2} (\boldsymbol{...
0
votes
1answer
37 views

Find the extremas of $f(x,y,z)=xyz-x^2-y^2-z^2$

Find the extremas of $f(x,y,z)=xyz-x^2-y^2-z^2$. After some calculation, $Df(x,y,z)=0$ for and only for $$ \\p_1=(0,0,0), p_2=(2,2,2),p_3=(-2,-2,2),p_4=(-2,2,-2),p_5=(2,-2,-2)\ \\ H(f)=\begin{vmatrix}...
0
votes
0answers
23 views

Trouble calculating the Laplace-Beltrami operator through this formula

Let $U$ be an open, bounded and connected subset of $\mathbb R^3$ with a $C^2−$regular boundary $\partial U$. For an arbitary $x_0 \in \partial U$ define the function $f:B(x_0,r) \cap \partial U \to \...
1
vote
0answers
18 views

How to classify the degenerate stationary points of a multivariate function?

I have a multivariate function in one of whose critical points the Hessian matrix is singular. Is there any general method to determine the type of this critical point? Would be worth plotting the ...
0
votes
1answer
31 views

How to study the critical points of a $2$-variable function?

I am revising some past exam questions and there is one that states: Study the critical points of the function: $$f(x,y)=x^2+y^2-x^4-y^4-2x^2y^2.$$ According to my professor, this is what I have ...
0
votes
0answers
27 views

Hessian matrix is (strictly) negative definite at some point

Give a smooth function $f(x,y)$ on $\mathbb{R}^2$ and let $D = [0,1]\times[0,1]\subset \mathbb{R}^2$. Suppose $f$ is negative on the boundary $\partial D$ but reaches a positive maxima in the interior ...
0
votes
1answer
15 views

Prove $y^tH_f(a)y \leq 0$ with Taylors Theorem

Let the function $f \in C^2(\mathbb{R}^n;\mathbb{R})$ have a local maximum in the point $a \in \mathbb{R^n}$. How can one prove the following with Taylor's theorem: The following applies: $y^tH_f(a)...
3
votes
1answer
66 views

Is there a systematic way to construct functions with prescribed local extrema?

I'm teaching multivariable calculus and having a hard time coming up with optimization problems. Suppose I have three lists of points $\{a_1, \dotsc, a_r\}$, $\{b_1, \dotsc, b_s\}$, and $\{c_1, \...