6
$\begingroup$

Define

$$x'=\text{argmin}_{x_1}f(x_1,\lambda),$$

where $f$ is a strictly convex function on $x_1$ and $\lambda$. I would like to ask if there is any theorem about the continuity of $x'$ w.r.t $\lambda$? If yes, can it be generalized to higher dimemions?

For example, I have

$$\mathbf{y}'=[y_1' ~~ y_2']^T=\text{argmin}_{y_1,y_2}g(y_1,y_2,\lambda),$$

where $g$ is a strictly convex function on $y_1$, $y_2$ and $\lambda$. Are $y_1'$ and $y_2'$ continuous w.r.t to $\lambda$?

$\endgroup$
  • $\begingroup$ This is clearly going to depend on the regularity of $f$, respectively $g$. A first rough result would be that if $f$ is $\lambda$-continuous in a an open set $U$, and it is strictly convex for any $\lambda \in U$, then $x'$ is $\lambda$-continuous in $U$; if it wasn't we would have a value of $\lambda$ for which $f$ has two different minimums.On the other hand if $f$ is not $\lambda$-continuous $x'$ doesn't have to be too. But I guess you are looking for something a bit more refined, right? Do you have any specific statement in mind? $\endgroup$ – Giovanni De Gaetano Feb 10 '15 at 9:27
  • $\begingroup$ Chapter 7.E in Rockafellar & Wets, Variational Analysis deals with this. $\endgroup$ – Ronaldo Carpio Feb 11 '15 at 3:54
  • $\begingroup$ @Giovanni De Gaetano yes. g(y_1,y_2,\lambda)=a(y_1,y_2)+\lambda b(y_1,y_2), where a is strictly convex and b is convex. Could you suggest any theorem about the continuity of the argmin to \lambda? $\endgroup$ – Timespace7 Feb 11 '15 at 14:13
  • $\begingroup$ In your setting, without the additional assumption that $g(y_1,y_2,\lambda)$ is strictly convex, I don't even see why $\mathrm{argmin}$ should be a well defined function. In general it is defined to be the set of points where the argument achieves a minimum. If you could provide some more background perhaps I could be more helpful. And, as a side remark, if you enclose your latex code in dollar symbols $ it is going to look at it's supposed to. $\endgroup$ – Giovanni De Gaetano Feb 11 '15 at 16:02
3
$\begingroup$

I think that you need to carefully state the conditions on the function $f$. There is not enough information, for instance, to ensure that the function is even defined: You need conditions ensuring that there is a minimum for each $\lambda$. Convex functions are very general creatures, and need not even be continuous.

However let us say that $f(x,\lambda):\mathbb{R}^{m+n}$ is 2-times continuously differentiable with respect to the variable $(x,\lambda)$ with $x\in\mathbb{R}^m$ and $\lambda\in\mathbb{R}^n$.

Say that for $\lambda^1$ that there is a point $x^1$ such that $\nabla_x f(x^1,\lambda^1)=0$ and the Jacobian with respect to $x$ at this point $\frac{\partial f}{\partial x}$ is invertible. This is a minimum since $f$ is convex. Then there exists a neighbourhood $V$ around $\lambda^1$ such that we can we can define a continuously differentiable function g:

\begin{align} g(\lambda) &\triangleq x \text{ such that $\nabla_x f(x,\lambda) = 0$} && \text{(implicit function theorem)}\\ &=\text{argmin}_{x\in\mathbb{R}^m}f(x,\lambda) &&\text{(by convexity)} \end{align}

This all follows from the implicit function theorem applied to the function $\nabla f(x,\lambda)$ and the equation $\nabla f(x,\lambda)=0$. I.e. yes, this argmin function is continuously differentiable under some circumstances.

The Jacobian $\partial_i \partial_j f$ needs to be invertible at this minimum $x_1$. Since $f$ is a convex function, the jacobian is symmetric nonnegative. The Jacobian will be invertible if it is positive definite: I.e. if $f$ is strongly convex in a convex neighbourhood of $x_1$ for $\lambda=\lambda_1$.

If you want to use weaker conditions, I'll see if I can find another theorem.

$\endgroup$
  • $\begingroup$ Amazing answer...but can you find weaker conditions? I'd be very interested in this also... $\endgroup$ – AIM_BLB May 3 '17 at 14:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.