1
vote
1answer
35 views

Distributed Convex Optimization Algorithm

Consider the convex optimization problem $$ \min_{x_1, \cdots, x_N, y} \sum_{i=1}^{N} f_i(x_i,y) $$ $$ \text{subject to: } x_i \in X_i \ \ \forall i, \ \ y \in Y, \ \ y = \sum_{i=1}^{N} x_i $$ ...
0
votes
1answer
26 views

Reentrant constraints in active set algorithm?

Problem definition Supposing you're trying to solve a quadratic program: $$ \min_x f(x) = \frac{1}{2}x^T Q x + c^T x \\ \mbox{s.t} \, \; A x \ge b$$ Where Q is square ($n$x$n$), positive semi ...
-2
votes
0answers
19 views

Quadratic programming with linear equality constraints in Matlab [on hold]

I have to identify an ARX under some linear constraints, this means that I have a quadratic programming with linear equality constraints problem. One way is to use the following equations in the ...
2
votes
2answers
38 views

Augmented Lagrangian

Consider the following equality constraint minimization problem: minimize $\text{ }f(x)$ subject to $Ax=b$ Its Lagrangian is then: $L(x,y) = f(x) + y^T(Ax-b)$ We can use then gradient ascent to ...
0
votes
3answers
42 views

Optimizing an expression containing sum of square roots of squared terms

For optimization problems involving square root, it is common to optimize the squared expression instead of that containing the square root. What if we have sum of squared expressions ? Consider the ...
6
votes
1answer
104 views
+50

Intuition behind accelerated first-order methods

$\newcommand{\prox}{\operatorname{prox}}$ $\newcommand{\argmin}{\operatorname{argmin}}$ Suppose that we want to solve the following convex optimization problem: $\min_{x \in \mathbb{R}^n} g(x) + ...
0
votes
0answers
46 views

Convex optimization approximation

Consider the optimization problem $\mathcal{P}_0$ $$ \min_{x \in \mathbb{R}^2} \left\| x-p \right\|^2 $$ $$ \text{sub. to: } \ A x \leq b, \ \ x_1^2 + x_2^2 = 1 $$ where $p \in \mathbb{R}^2$ is a ...
0
votes
1answer
45 views

Model $\min \frac{1}{2} \parallel Ax-B \parallel_2 + \lambda_1 \parallel Cx \parallel_1 + \lambda_2 \parallel Dx \parallel_\infty $ into standard form

I need to solve the following convex optimization problem: $\min \frac{1}{2} \parallel Ax-B \parallel_2 + \lambda_1 \parallel Cx \parallel_1 + \lambda_2 \parallel Dx \parallel_\infty$ s.t $x ...
2
votes
3answers
95 views

Is this optimization problem solvable?

I have the following optimization problem: $$ \min_\mathbf{x}\|\mathbf{a+Bx}\|^2 ~~\text{s.t}~~ \|\mathbf{y+Ax}\|_\infty \leq \beta\|\mathbf{y}\|_\infty ~~,~~ \|\mathbf{x}\|^2 \leq \alpha^2$$ where ...
1
vote
2answers
46 views

How to solve this optimization problem? (may be gradient descent?)

I have the following optimization problem. $$\operatorname*{argmax}_{w} \|(1-w)\boldsymbol{X} -w\boldsymbol{Y}\|^2 \\ s.t. \quad 0<w<1 $$ How can I find the solution of this problem? May be ...
1
vote
0answers
38 views

Alternating Direction Method of Multipliers (ADMM) application

$\newcommand{\argmin}{\operatorname{argmin}}$ Recall, that ADMM algorithm solves the problem of the form: $\min \text{ } f(X) + g(Z)$ $\text{s.t. } AX + BZ = C$ where $X$, $Z$ and $C$ are real ...
1
vote
1answer
103 views

Knuth's Sandwich Theorem: requesting proof clarification

The question is about F6 of Section 8 ("Elementary facts about cones") in Donald Knuth's Sandwich Theorem (http://arxiv.org/pdf/math/9312214.pdf). He claims to prove $(A \cap B)^* = A^* + B^*$ when ...
0
votes
1answer
25 views

How to deal with non-existent derivatives in Lagrangian?

I am stucked at a detail in a constrained optimization problem: Question Assume that the objective function is continuous on its domain $D$, but at some points $Z \subseteq D$ it is not ...
0
votes
1answer
25 views

Coordinate descent with constraints

Coordinate descent is a powerful method for solving optimization problems like $$\min_x \tfrac{1}{2}x^T A x + b^T x + \lambda ||x||_1$$ where $A$ is symmetric and positive definite, $\lambda>0$ ...
-1
votes
0answers
32 views

Constraint optimization with lagrangian

I am having trouble regarding the general steps one needs to take in order to solve an constraint optimization using Lagrangian. More specifically, I want to maximize objective equation $f(x,y,z,w)$ ...
0
votes
3answers
160 views

How exactly do I prove that I find the maximum of the function

I am currently trying to maximize an objective function $f(a,b,c,d,e)$ over the variable $b$ only. By taking the derviative of f over b, setting it to zero, I can solve b in terms of the other 4 ...
3
votes
0answers
38 views

Strong duality: When does the optimal primal variable coincide with the primal variable giving the dual function.

I'm considering the inequality-constrained optimization problem of finding $$ x^{\star} = \arg \min_{x} f(x) \;\; \text{s.t.} \;\; h(x) \le 0 $$ which is assumed to have a unique minimizer. The ...
1
vote
2answers
44 views

L1 regularized SVM in Matlab

Minimizing the following SVM formulation \begin{align} \arg\min_{\mathbf{w}}\frac{1}{2}\|\mathbf{w}\|^2_2 \\ \text{subject to } \quad y_i(\mathbf{w}\cdot\mathbf{x_i}) \ge 1 \end{align} can be done ...
0
votes
1answer
34 views

Reference for gradient descent with unit norm constraint

I faced a non-convex optimization problem with unit norm constraint. I can solve the problem using the gradient descent method and the projection of the gradient onto the tangent plane as in @joriki ...
1
vote
2answers
33 views

multi-objective optimization

I am currently encounterring a optimization problem. The goal is optimize an objective function A and B at the same time. But the problem is that optmizing A will almost always tradoff with B, such ...
2
votes
1answer
42 views

Origin of Slater's condition

I've been looking all over the internet to answer this question: Slater's condition is a commonly used to certify that strong duality holds in a convex optimization problem. Although used in many ...
3
votes
2answers
106 views

What numerical methods are known to solve $L_1$ regularized quadratic programming problems?

What numerical methods are suitable to solve the following problem $$\min_x \tfrac{1}{2}x^T A x + b^Tx + \lambda ||x||_1$$ where $x,b\in\mathbf{R}^n$, and $A\in \mathbf{R}^{n\times n}$ is positive ...
0
votes
1answer
35 views

Strong convexity of a function with cases

Given a set $S = \{x_1,\dotsc,x_n\} \subset \mathbb{R}$, is the function \begin{align} f&: (0,\infty) \to \mathbb{R} \\ f&(p) = 2p^2 + \frac{1}{n}\sum_{i=1}^n \max(0, -p^2-x_i) \end{align} ...
0
votes
0answers
28 views

dual value of a linear constraint

Assume a minimization problem. The dual of an inequality '<' constraint is the marginal improvement in the objective function (ie marginal reduction) by marginally increasing the right-hand-side ...
1
vote
1answer
57 views

New variable in a convex optimization problem

Consider the convex optimization program $$ \min_{x \in X } x^\top P x + p^\top x \quad \text{ sub. to: } Ax = b $$ where $X \subset \mathbb{R}^n$ is compact, $P \succ 0$, $A \in \mathbb{R}^{m \times ...
0
votes
1answer
32 views

Sums of positive and negative distances to the least squares plane

Let $A_{1}, A_{2}, \ldots, A_{n}$ be points in $\mathbb{R}^{3}$ and $\pi_{*}$ be the least squares plane, i. e. $$ \sum \limits_{i = 1}^{n}\rho^{2}(A_{i}, \pi_{*}) = \min_{\pi}\sum \limits_{i = ...
0
votes
0answers
26 views

Primal-dual subgradient method

In these notes, an extension of the subgradient method is presented in Section 8 (page 30). The method is described so quickly and neither convergence analysis (compared to classical subgradient for ...
1
vote
2answers
75 views

Formulation and computation of “the” unique median of an even-sized list

Consider an even-sized set of numbers $X = \{x_k\}$, such as $X = \{1, 2, 7, 10\}$. The median $m$ is defined as: $$m = \mathrm{arg \min_x} \sum_k \lvert x_k - x\rvert^1$$ Any $m \in [2, 7]$ is a ...
0
votes
0answers
28 views

Proof of sufficient condition of existence of Lagrange multipliers

Consider the optimization problem $$ (P) \quad \inf\{ f(x) : g_i(x) \le 0, i = 1,\ldots, m, x \in \mathbb R^m \} $$ where $f, g_i : \mathbb R^n \to (-\infty, \infty]$ are convex and $0 \ne ...
0
votes
1answer
34 views

Sufficient condition on convex function such that $f(x) > -\infty$ for all $x$.

Let $f : \mathbb R^n \to [-\infty, \infty]$ convex and let $f(\overline x) > -\infty$ for $\overline x \in \mbox{int}(\mbox{dom}(f)$. Show that $f(x) > -\infty$ for all $x \in \mathbb R$. ...
0
votes
1answer
25 views

Hessian of non-differentiable function

Given a function $f = \max\{f_1,f_2\}$ with $f_1,f_2$ convex and differentiable, I know I can calculate the subgradient of $f$. Is there also an equivalent of the subgradient for the (sub)Hessian? ...
2
votes
1answer
57 views

Solving $ \inf \left\{ F[\nu] : \nu \in L^2 , \nu \geq 0, \int _0 ^1 \nu=1\right\}$

Let $\phi \in \mathcal ( [0,1]^2)$ symetric , can we find a solution to the following minimisation problem? $$ \inf \left\{ F[\nu] : \nu \in L^2 , \nu \geq 0, \int _0 ^1 \nu=1\right\}$$ with $$ ...
0
votes
1answer
28 views

Dual norm equivalence?

$\|\|$ is a norm in $R^n$, its dual norm is defined as $\|s\|^*=max_{\|x\|=1}s^Tx$. We denote $s^\#$ as any vector in the following set: [Arg $max_x: \ \ s^Tx-\frac{1}{2}\|x\|^2$] How to verify ...
0
votes
1answer
25 views

How to solve the dual problem of SVM

By solving the primal form of SVM (support vector machine), we can get the dual form of this problem. The more details are shown in wiki of SVM. Given this dual problem, how can I solve the ...
1
vote
1answer
22 views

The Dual problem of a non constraints problem?

The primal problem is $min_{w\in R^d}: P(w)$ where $P(w)=\frac{1}{n}\sum_{i=1}^n\phi_i(w^Tx_i)+\frac{\lambda}{2}||w||^2$. The dual problem is $max_{\alpha\in R^n}: D(\alpha)$ where ...
1
vote
3answers
24 views

Constraint to unconstraint optimization problem by subsitution

Given the following convex optimization problem $\min_{x,p} ||x|| - p$ subject to $p > 0$ Can I change the above to an unconstrained convex optimization problem by substituting $c = ...
1
vote
1answer
26 views

Could I get the explicit solution to the following problem relate to generalized rayleigh quotient?

$\bf x$ and $\bf a$ are complex vectors, $\bf C$ is positive definite complex matrix, $\bf B$ is positive-semidefinite complex matrix. What's the objective value? Thanks! $$\max_{\bf x} ...
1
vote
0answers
27 views

First order necessary conditions for nondifferentiable nonconvex minimization problem

I am interested in first order necessary conditions for the following minimization problem where the function $f$ is continuous, nondecreasing and concave, with $f(0)=0$, but not necessarily ...
2
votes
1answer
27 views

How to prevent a convex optimization from being unbounded?

I'm novice in optimization and have a convex optimization function of form $\sum_{i,k} p_{k,i}*\log{p_{k,i}} $ to minimize with the following constraints: $\forall i, a_i = \sum_{k=1}^{m} b_k. ...
0
votes
1answer
21 views

What exactly is non-convex optimization

I am coming across the term: non-convex optimization problem. What exactly is this non-convex structure, and how do I know by only looking at the structure of the problem, I could tell it is ...
1
vote
1answer
34 views

Optimization of a convex target function with inequality constraints

I want to solve the following optimization problem: \begin{equation} \begin{split} \text{maximize} &\;\;\; \ln x_1+\ln x_2+\ln x_3+\ln x_4 \\ \text{s.t} &\;\;\; x_4\le4 \\ ...
-2
votes
1answer
67 views

Matlab optimization toolbox vs. CVX solver?

I would like to know what is the difference between the Matlab optimization toolbox and CVX solver which is a convex optimization toolbox? Can a convex optimization be solved in both?
0
votes
2answers
55 views

Regularization vs. Inequality Constraint

For what values of a regularization parameter $\alpha$, there is an equivalent inequality constraint in convex optimization? In particular, in the convex optimization problems below $$ \text{ Problem ...
0
votes
0answers
28 views

Finding a solution using the principle of maximum entropy?

I have set of linear constraints and would like to find an answer to its unknown variables, $p_i$'s. One of my options to find a solution for $p_i$'s using maximum entropy problem, $\max(\sum - p_i ...
0
votes
0answers
90 views

Maximize the maximum Eigenvalue under a diagonally constrained matrix

Suppose we have $N\times N$ Hermitian matrix $\mathbf{A}$ I want to find the real $N\times N$ diagonal matrix $\mathbf{D}$ that maximizes the sum of the maximum Eigenvalues : $\mathbf{D}=\arg\max ...
2
votes
2answers
82 views

How to maximize an entropy function?

I'm very novice in optimization and have a convex optimization function of form $\sum_{i,k} p_{k,i}*\log{p_{k,i}} $ to minimize with the following constraints: $\forall i, a_i = \sum_{k=1}^{m} b_k. ...
3
votes
1answer
58 views

Are all non-convex problems created equal?

The distinction between convex and non-convex problems is usually dubbed as the distinction between easy and hard problems. While in the convex case you are golden (local optima are global optima; ...
0
votes
0answers
25 views

Prove that dual variables become free variables

Let P: $max\ c^T x$ subject to $Ax\leq b $ Say if we replace the latter part by $Ax=b$. Show the effect on dual problem is that the variables of dual become free variables. Can you break Ax=b ...
0
votes
1answer
33 views

regarding the concept of dual cone

When studying the covex analysis, I am not clear about the concept of dual cone. In the following graph, $\mathcal{K}*$ was the dual cone. I marked two points, the ...
0
votes
1answer
74 views

Convex optimization: affine equality constraints into inequality constraints

I have the following problem: \begin{equation} \begin{array}{cll} \displaystyle \min_{ \mathbf{x} } & & \displaystyle f(\mathbf{x}) \\ \mathrm{s.t.} & & \mathbf{x} \in \mathcal{C} \\ ...