# Is the Hessian matrix of Euclidean norm function positive definite?

I defined an Euclidean norm function with a smooth technique to avoid not differentiable problem in point $$0$$, $$f: \mathbb{R}^n\to\mathbb{R}$$, $$f(x) = \sqrt{\|x\|^2+\epsilon^2},$$ where $$\epsilon=0.00001$$.

I know for sure that $$f(x)$$ is convex, so $$\nabla^2 f(x) \ \forall x\in \mathbb{R}^n$$ positive semi-definite.

My question: is the Hessian matrix $$\nabla^2 f(x) \ \forall x\in \mathbb{R}^n$$ positive definite?

I scaled down the problem to 1-dimension, such that we can write down the gradient(first derivative) and the Hessian( derivative) easily, $$f'(x) = \frac{x}{\sqrt{(x^2+\epsilon^2)}}$$ $$f''(x) = \frac{\epsilon^2}{(x^2+\epsilon^2)\sqrt{(x^2+\epsilon^2)}}.$$

It follows $$f''(x)>0 \ \forall x\in\mathbb{R}$$, which make it a positive definite matrix.

Can we generalize it to high dimensions $$n$$?

• You can calculate $\nabla^2 f(x) = (\lVert x \rVert^2 + \varepsilon^2)^{-\frac{1}{2}}I - xx^\top(\lVert x \rVert^2 + \varepsilon^2)^{-\frac{3}{2}}$ Jul 29 at 22:12
• If you use the euclidean norm, then for $x=(x_1,...,x_n)^T$, we have $f(x)= \left\|\pmatrix{x_1\\\vdots\\ x_{n}\\\epsilon}\right\|$. Now use $\|x\|= \sqrt{x^T x}$. Calculate $\nabla^2 f$ and substitute in $\epsilon$. Notice that you get a sum of a rescaled identity matrix and a rank-1-matrix. Use the Matrix determinant lemma to calculate the determinant of all relevant minors. Jul 29 at 22:44

$$\def\e{\varepsilon} \def\l{\left} \def\r{\right} \def\p{\partial} \def\lr#1{\l(#1\r)} \def\trace#1{\operatorname{Tr}\lr{#1}} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\hess#1#2#3{\frac{\p^2 #1}{\p #2\,\p #3^T}}$$Let $$x$$ be a vector in $${\mathbb R}^{n}$$.
Square the function before calculating the differential and gradient of $$f$$ \eqalign{ f^2 &= x^Tx + \e^2 \\ 2f\,df &= 2x^Tdx \\ df &= \lr{\frac{x}{f}}^Tdx \\ \grad{f}{x} &= \frac xf \;=\; g &\quad\quad\big({\rm Gradient\,vector}\big) \\\\ } Now calculate the differential and gradient of $$g$$ \eqalign{ dg &= \frac{f\,dx-x\,df}{f^2} \\ &= \frac{fI\,dx-x\lr{\frac xf}^T\,dx}{f^2} \\ &= \lr{\frac{f^2I-xx^T}{f^3} }dx \\ \grad{g}{x} &= \frac{f^2I-xx^T}{f^3} \;=\; H &\qquad\big({\rm Hessian\,matrix}\big) \\ } Note that for vectors orthogonal to $$x$$, the action of the Hessian matrix is simple linear scaling $$Hv = {\frac{v}{f}} \qquad\qquad\qquad\qquad\qquad$$
Vector components which are parallel to $$x$$ gets are scaled more aggressively. In the limiting case $$Hx = {\frac{\e^2x}{f^3}} \qquad\qquad\qquad\qquad\qquad$$
So setting $$\,\e^2=0\;$$ would make $$Hx=0,\;$$ and the matrix would be semi-definite.
However, since $$\e^2>0\;$$ the matrix is positive-definite.