From Boyd & Vandenberghe's book, page 491:

Applying the Lipschitz condition,we have $$\begin{aligned} \left\|\nabla f\left(x^{+}\right)\right\|_{2} &=\left\|\nabla f\left(x+\Delta x_{\mathrm{nt}}\right)-\nabla f(x)-\nabla^{2} f(x) \Delta x_{\mathrm{nt}}\right\|_{2} \\ &=\left\|\int_{0}^{1}\left(\nabla^{2} f\left(x+t \Delta x_{\mathrm{nt}}\right)-\nabla^{2} f(x)\right) \Delta x_{\mathrm{nt}} d t\right\|_{2} \\ & \leq \frac{L}{2}\left\|\Delta x_{\mathrm{nt}}\right\|_{2}^{2} \\ &=\frac{L}{2}\left\|\nabla^{2} f(x)^{-1} \nabla f(x)\right\|_{2}^{2} \\ & \leq \frac{L}{2 m^{2}}\|\nabla f(x)\|_{2}^{2} \end{aligned}$$ to get the inequality assumption (9.33), which is:

$$\frac{L}{2 m^{2}}\left\|\nabla f\left(x^{(k+1)}\right)\right\|_{2} \leq\left(\frac{L}{2 m^{2}}\left\|\nabla f\left(x^{(k)}\right)\right\|_{2}\right)^{2}$$

I dont understand how he get the first two lines of the derivation?

My understanding is that $\nabla f\left(x^{+}\right)$ equals $\nabla f\left(x+\Delta x_{\mathrm{nt}}\right)$ already when t=1, so where do the rest of two terms come from? please anyone help me, really appreciate.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.