0
$\begingroup$

Given that $$g(\theta):= f_1(\theta_k)+\nabla f_1(k)^T(\theta -\theta_k)+\frac{L}2 \|\theta -\theta_k\|_2^2 + f_2(\theta) $$ Define $\theta_{k+1}$ to be the minimizer of $g(\theta)$, show that $$\theta_{k+1} = arg \ min_\theta\;\Big\{{f_2(\theta)}+\frac{L}{2} \|\theta - \theta_k +\frac{1}L \nabla f_1(\theta_k)\|_2^2 \Big\}$$

Note that, $f_1(\theta)$ is convex and $\nabla f(\theta)$ is Lipschitz continuous that existing a constant $L$ such that $\|\nabla f(\theta_1)-\nabla f(\theta_2)\|=L\|\theta_1-\theta_2\|$ . $f_2(\theta)$ is a semi-continuous convex function.

$\endgroup$

1 Answer 1

0
$\begingroup$

First it is clear that by convexity of $f_1$ and $f_2$, then $g(\theta)$ and $g_2(\theta):=f_2(\theta)+\frac{L}{2}\|\theta-\theta_k+\frac{1}{L}\nabla f_1(\theta_k)\|$ are strictly convex functions. Therefore, by l.s. continuity inherited from $f_1$ and $f_2$ they attain their minimum, which is unique by strict convexity.

Assume that $\theta$ is chosen such that $\theta=arg min(g(\theta))$. It is equivalent to $0=\nabla g(\theta)$ (by convexity) so $\theta$ is such that $$ 0=\nabla f_1(\theta_k)+\nabla f_2(\theta)+L(\theta-\theta_k) $$

Furthermore, by convexity finding the minimum of $g_2$ is equivalent to find a solution of $0=\nabla g_2(.)$. So, that $$ 0=\nabla f_1(\theta_k)+\nabla f_2(\theta)+L(\theta-\theta_k) $$

It follows that a minimum of $g(\theta)$ is also a minimum of $g_2(\theta)$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .