This question is an exact duplicate of:

I was reading this paper http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2012_0388.pdf

and I came across this part

enter image description here

I didn't get how the third line came from the second line. Any suggestions?


marked as duplicate by Michael Grant, Avitus, Hakim, Gabriel Romon, user61527 Jun 19 '14 at 22:16

This question was marked as an exact duplicate of an existing question.


The $H_k$-norm is defined as $$ \|x\|_{H_k}^2 = x^TH_kx. $$ Hence, we obtain $$\begin{split} \|(y-x_k) + H_k^{-1} \nabla g(x_k)\|^2 &= ((y-x_k) +H_k^{-1} \nabla g(x_k))^TH_k((y-x_k) +H_k^{-1} \nabla g(x_k))\\ &=(y-x_k)^TH_k(y-x_k) + 2 (y-x_k) ^T\nabla g(x_k) + \nabla g(x_k)^TH_k^{-1}\nabla g(x_k). \end{split}$$ Observe that the last addend is independent of $y$, so \begin{multline*} \arg \min_y\left\{(y-x_k)^TH_k(y-x_k) + 2 (y-x_k) ^T\nabla g(x_k) + \nabla g(x_k)^TH_k^{-1}\nabla g(x_k)\right\}\\=\arg \min_y\left\{(y-x_k)^TH_k(y-x_k) + 2 (y-x_k) ^T\nabla g(x_k) \right\}. \end{multline*}


Not the answer you're looking for? Browse other questions tagged or ask your own question.