I need to prove the following statement:

Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ be a $C^2$ function. Suppose a sequence $(x_k)$ converges to $x_*$, where the Hessian $Hf(x_*)$ is positive definite. Let $\nabla f_k := \nabla f(x_k) \neq 0$, $H f_k := Hf(x_k) \neq 0$, $d_k: = - B_k^{-1} \nabla f_k$, and $d_k^N := -[Hf_k]^{-1} \nabla f_k$ for each $k$, where each matrix $B_k$ is symmetric and positive definite.

I need to show that:

$$ \lim_{k \to \infty} \frac{\|(B_k-Hf_k)d_k\|}{\|d_k\|}=0\iff \lim_{k \to \infty} \frac{\|d_k - d_k^N\|}{\|d_k\|}=0.$$

I have no idea where I should start and how to proceed. Could you just provide with me some clues?

Note: $\|\cdot\|$ is $2$-norm.


1 Answer 1


In relation to the two sequences of norms you introduced above let us define:

$$t_n=\nabla{f}(x_n), \mathrm{H}f(x_n)=H_n, \alpha_n=\left\lVert B_n^{-1}t_n\right\rVert, u_n=\frac{(H_nB_n^{-1}-\mathrm{I}_n)t_n}{\alpha_n}, v_n=\frac{(B_n^{-1}-H_n^{-1})t_n}{\alpha_n}$$ for arbitrary $n \in \mathbb{N}$. You must prove that $u$ converges to $0_n$ if and only if $v$ does so as well. Notice that $$u_n=H_nv_n$$

for all $n \in \mathbb{N}$. Under the assumption that $v$ converges to $0_n=(0)_{1 \leqslant k \leqslant n}$, it would suffice to show that the sequence of matrices $(H_n)_{n \in \mathbb{N}}$ is bounded in the operator norm which we must not forget is a norm equivalent to the euclidean one on the space of matrices. From your conditions in the hypothesis can you ascertain why this should be the case?

The reasoning above will have taken care of one half of the equivalence. As to the remaining implication, it would be most useful if the terms of $v$ were also expressible as matrices multiplying those of $u$, which leads us naturally to ask two questions:

  1. Are the matrices $H_n$ invertible, at least starting from a certain rank $k \in \mathbb{N}$ onwards?

  2. If the answer to 1) is affirmative and we can thus write $v_n=H_n^{-1}u_n$ for $n \geqslant k$, is there any reason why the sequence $\left(H_n^{-1}\right)_{n \geqslant k}$ would be bounded (in any of the usual norms, as they are all equivalent)?

I hope this will be of use to you.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .