2
$\begingroup$

I didn't find any information on the internet about the globalized $\textit{BFGS}$ method. Wikipedia only talks about the normal BFGS, without this globalized condition:

Assume direction of descent $d = -H_f^{-1}(x^k)\cdot \nabla f(x^k)$. Instead of calculating the inverse of the Hessian matrix we use an approximation method with two Rank-1 Updates, the result of this method is the Matrix $B^{-1}$.

Now, the condition of the globalized BFGS method:

$\nabla f(x^k)^T\cdot d \leq -\rho||d||$ with $\rho$ being a constant (e.g. $0.001$)

If we happen to have this condition unfulfilled (this would probably be the result of a wrong Hessian-Matrix approximation), we choose $d = -\nabla f(x^k)$ for this step and also reset the BFGS method by either:

$\circ$ (1) Setting $B^{-1} = I$ for our next iteration

$\circ$ (2) Doesn't do anthing leave $B^{-1}$ as is for next iteration

I don't really see a difference in (1) or (2). Assume we would choose (1) for our implementation, due to the formula $d = -B_f^{-1}(x^k)\cdot \nabla f(x^k)$ with $B = I$ would result in $d = -\nabla f(x^k)$.

With (2) - how can we be sure that we are able to fullfill said condition again ever? Assume we never update $B^{-1}$ again because our $d$ doesn't fulfill the condition, we would always put $d = -\nabla f(x^k)$. In a different scenario we might be able to fulfill the condition in step $k + 10$, now we are doing the two Rank-1 updates, but with a matrix $B$ from step $k$ - so $10$ steps ago! How could that work ever!?

All in all I don't really get this resetting or globalized BFGS method.

I would really appreciate if someone could explain to me why $(1)$ or $(2)$ would be a good choice.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.