Convex Optimization is a special case of mathematical optimization. It includes Linear Programming and least-squares.

learn more… | top users | synonyms

4
votes
0answers
103 views

How to find accuracy of Matlab's quadprog solver?

I have solved with quadprog from Matlab a strong convex quadratic problem given as $$ f(x) = x^TQx + c^Tx$$ with constrains $$ Cx \leq b.$$ Now the output of quadprog is: Minimum found that ...
0
votes
1answer
28 views

unnecessary constraint in optimization problem

I have some optimization problem (optimizing parameter $\alpha$)with those constraints: $$\alpha_i\ge0$$ $$\sum\limits_i \alpha_i y_i =0$$ and a third constraints: $$w-\sum\limits_i \alpha_i y_i x_i = ...
1
vote
0answers
57 views

Is there any way to transform a non-convex optimization problem into a convex one?

I have an optimization problem which is described as $$\begin{array}{ll} \text{minimize}_x & c^{T}x\\ \text{subject to} & Gx \preceq h\\ & -x^{T}Px - qx - r \leq 0 \end{array} $$ where ...
2
votes
0answers
37 views

Effective convexity criterion for the finite point set in $\mathbb{R}^3$

I need to find effective convexity criterion for the finite point set. Below there is description of what is meant by "effective" criterion. Definition. Let $M = \{A_{1}, \ldots, A_{n}\}$ be the ...
1
vote
0answers
30 views

Is there a good textbook/book out there that explains sub gradients thoroughly?

I was interested in learning and understanding sub gradients as much as I could from some good resource. I know what the definition is, but I seem unable to apply the definition to prove basic facts ...
0
votes
1answer
49 views

What is the sub-differential of the separable sum $R(w) = \sum^{d}_{j=1} |w_j|$?

Recall the definition of a sub-differential: $$\partial F(w_0) = \{ v : \forall w, F(w)-F(w_0) \geq v \cdot (w - w_0)\} $$ Intuitively, for any w in the domain of the function one can draw a plane ...
1
vote
1answer
33 views

How does $\in$ behave with simple algebra dealing with sub gradients?

I was trying to understand the following optimization problem: $$argmin_{v \in H} {R(v) + \frac{1}{2}||v - w||^2}$$ Assume $R(v)$ is Convex, proper and semi-continuous with a unique minimizer. ...
0
votes
1answer
25 views

Testing for Convexity

Could somebody please explain the method for answering a question like this?
1
vote
3answers
195 views

Compact set in R that is not convex?

Just need an example. For example, the I know the set [0,1] is compact because it is obviously closed and bounded. But I have no idea how to test for convexity
1
vote
1answer
57 views

Strong convexity on sets?

Consider the definition of convex functions: $$ f(tx+(1-t)y) \le t f(x)+(1-t)f(y) $$ It is easy to show the definition of the convexity on sets with respect to the above definition (Specifically for ...
0
votes
0answers
23 views

A variant of submodularity?

See the definition of submodulation functions: $$ f(A) + f(B) \geq f(A \cup B) + f(A \cap B) $$ Suppose I make this definition a little stronger: $$ f(A) + f(B) \geq f(A \cup B) + f(A \cap B) + A ...
0
votes
1answer
69 views

Subgradient example

Let $f(x_1, x_2)$ be defined as: if $x_1 =0$ then $f(x_1,x_2)=x^2_2$ else $\infty$ The subgradient of $f(x_1,x_2)$ at $(0,0)$ is given as: $\mathbf{R} \times \{0\}$. (The real line crossed with ...
0
votes
2answers
41 views

Prove $A^TB$ is a positive semi-definite matrix?

$A,B \in R^{m\times n}$ and the singular values of both $A$ and $B$ are between 0 and 1. Is $A^TB$ a positive semi-definite matrix? Please show me the proof:)
1
vote
1answer
38 views

Linear independence of equality constraint gradients in constraint qualifications

I'm, trying to get an intuitive feel for the various constraint qualifications for KKT points. Most of them seem to rely on the linear independence of $\nabla g_i(x^*)$ where $g_i$ are the equality ...
1
vote
1answer
40 views

Difference between tangent cone and this constraint cone

Define the cone $G(\textbf{x} = \{\textbf{p} \in \mathbb{R}^n | \nabla g_i(\textbf{x}^T \textbf{p} \leq 0, i \in I(\textbf{x})\}$ So this is a cone associated with the inequality constraints that is ...
0
votes
0answers
36 views

constraint optimization: sparsity with non zero constraints

I have an obtimization problem in the following form. $\min f(x)\\ s.t \|x_i\|_0\leq\lambda\\ x_i \geq0\\ \sum_i x_i = 1$ where $f(x)$ is convex. What is the easy way to optimize it as I have a ...
2
votes
0answers
278 views

Affine functions as equality constraints in convex optimization problems

I am studying on an introduction to convex optimization problems. When defining a convex optimization problem, we have a convex object function, $f(x)$, a set of convex functions $g_i(x)$ where the ...
0
votes
0answers
26 views

Can the tangent cone contain more than the closure of the cone of feasible directions?

We know that the closure of the cone of feasible directions is contained in the tangent cone, but I'm wondering if it's possible for the tangent cone to consist of more than this closure?
2
votes
1answer
163 views

Is there a nice representation for KKT conditions for matrix constraints?

I have a convex programming problem: $\min \left\lVert J - R \right\rVert _F$ $J,R$ are matrices. $J$ is given for the problem. One of the constraints is: $R = KQ$ Here, $R,K,Q$ are matrices. $K$ ...
1
vote
2answers
44 views

$\,f:[0,1)\to\mathbb R$ is a concave differentiable function st $\,f(0)=0$. Show that $g:[0,1),g(z)=f(zx)/z$, for $x > 0$ is decreasing

Question: Suppose that $f : [0, 1) \to\mathbb R$ is a concave differentiable function such that $\,f(0) = 0$. Show that $g : [0, 1) \to\mathbb R$ defined by $g(z) = f(zx)/z$, for some given $x > ...
0
votes
1answer
49 views

distance between solutions in a convex optimization

Assume that you have the following convex optimization problem: $\min_{M} \|b+A\ M\ v\|_2$ subject to : $\|M\|_{2}<1$ (maximum singular value less than 1) where M is a suare matrix (n by n), A ...
0
votes
1answer
63 views

matrix convex optimization

How to solve the following problem explicitly? I mean closed form solution if possible. $\min_{M} \|M\ a-b\|_2$ subject to : $\|M\|_{\infty}<1$ (maximum singular value) where $M$ is a square ...
0
votes
2answers
147 views

Rank reduction to satisfy Barvinok's upper bound & Rank of a set notation

After reading n times the four first sections of the 4th Chapter of J.Dattorro's book (Convex Optimization & Euclidean Distance Geometry). I am confused between yes or no, every extreme point of ...
2
votes
2answers
113 views

Boundedness of sublevel sets of convex function (Boyd VandenBerghe)

(This is from the book Convex Optimization on p.474 on algorithms for unconstrained minimization) Assumptions The function $f : \mathbb{R}^N \mapsto \mathbb{R}$ is convex and twice-differentiable ...
0
votes
1answer
28 views

Confusion with a proof about the continuity of convex functions

I studying convex analysis and in my book I have the following statement and proof: Lets assume that $f:S\rightarrow \mathbb{R}, \;S\subset \mathbb{R}^n$ is a convex function. Then $f$ is ...
1
vote
1answer
61 views

Optimization with changing objective function

Is there any theory about (convex) optimization where the objective function is allowed to change during the optimization process? I have a problem where the objective function depends on some ...
1
vote
2answers
90 views

Why is any subspace affine?

I am studying 'Convex Optimization' written by Stephen Boyd. I am confused by an assertion in the book(page 27). Any one can tell me why and give an explanation ? Any subspace is affine, and a ...
1
vote
0answers
32 views

Partial concave maximization of subset of variables

Let $f(x_1, \dots, x_N)$ be a concave function in $x_1, \dots, x_N$. For arbitray $n>1$, prove that the (constrained) truncated function defined by $$g(x_1, \dots, x_{n-1}) = \max_{x_n, \dots, x_N ...
1
vote
2answers
64 views

Why does gradient descent make sense?

Suppose I define two functions of $x$ in terms of a convex function $f$ with a unique minimum $x_0$: $$f_1(x) = 1 \times f(x)$$ $$f_2(x) = 2 \times f(x)$$ Suppose I wanted to minimize each of these ...
0
votes
0answers
26 views

How to show that this function is finite?

I have the following function: $$Q(x) = \mathbb{E}_{\omega}\Big{[}v(h(\omega)-T(\omega)x)\Big{]}$$ with $\mathbb{E}[|\omega_i|]<\infty$ and $v(z)$ finite for all $z\in \mathbb{R}^m$. $v$ is a ...
0
votes
1answer
19 views

An optimal solution that is also smooth

I am given a vector x. My objective is to find an optimal y (minimize $||y-x||_2^2$). With the constraint $y(c) = a$ (a and c are known scalars). $$\text{minimize}_y ||y-x||_2^2 \\ \text{subject to}\ ...
1
vote
1answer
45 views

Optimisation over matrix entries

I was looking to write the KKT conditions to solve this optimisation problem. $$\min_{\substack{\sum_j x_{ij}\le k_i \\ i=1,2,\ldots N}} a^\top (I-X)^{-1} b $$ Since there are $N^2$ decision ...
0
votes
0answers
33 views

Projection onto a matrix where the diagonals are identity matrix

I'm trying the understand intersection of convex sets given in "Convex Optimization - Boyd" which I'm also trying to code in cvx. The two convex sets I'm trying to find the intersection are given ...
3
votes
0answers
54 views

The difference between affine set and affine hull

According to the definition of affine hull and affine set. $$aff [C] = [\theta_1x_1+...+\theta_nx_n|x_1,...x_n \in C, ...
1
vote
0answers
73 views

What is the left derivative of the hinge loss function in the context of subgradients?

Let: $$|a|_+ = max\{0,a\}$$ Then the Hinge loss function (in the context of classification in Machine Learning) is: $$V(-yf(x)) = |1 - yf(x)|_+$$ Note that $y \in \{-1,1\}$ Let $f(x) = \langle w, ...
0
votes
0answers
38 views

Second derivative of Bregman divergence

Suppose I define an exponential family distribution: $$ f(x; \theta) = \exp \left( \langle x, \theta \rangle - h(x) - \psi(\theta)\right) $$ where the log-partition function is: $$ \psi(\theta) = ...
1
vote
0answers
82 views

Proving/deciding concavity of a function of two variables

I would like to formally prove that the function $f(x,y) = \frac{(c+1)e^{-x}(xe^{x+y}+y)}{(c+2)(e^{x+y}-1)+e^y} $ is concave ($ c>2$ is a constant, and both $x,\, y \in \mathbf{R_+}$). Plots of ...
2
votes
1answer
68 views

Convex optimization: interpretation of the dual variable

Let us consider the convex optimization problem $$ \tag{P} \underset{x\in\mathbb R^n}{\sf minimize} ~~ f(x) ~+~ g({\bf L}x) $$ where ${\bf L}\in\mathbb R^{m\times n}$. Using the convex conjugate, ...
0
votes
0answers
57 views

gradient descent - cost reduces and then increases

I am optimizing a function using Gradient Descent. The learning rate is fixed. First for few iterations the cost decreases after that it starts increases. What is the reason for this?
3
votes
0answers
67 views

Normalize gradient

I want to minimize a function $f \, : \, \mathbb{R}^{N} \, \longrightarrow \, \mathbb{R}$ (with $N \in \mathbb{N}^{\ast}$. In my problem, $N = 315$). I know that $f$ is differentiable on ...
0
votes
1answer
29 views

Explanation for Zeroth Order Condition for Convexity

First of all, please let me admit that my math is very rusty so that I may not understand some basic concepts. I'm reading the book named "Convex Optimization" by Stephen Boyd and Lieven ...
1
vote
0answers
39 views

When result of max of min problem is equal to min of max problem

Let's assume there are two functions $f(x)$ and $g(x)$. I want to know when the optimal $x$ of max of min of $f(x)$ and $g(x)$ is not equal to optimal $x$ of min of max of $\frac{1}{f(x)}$ and ...
1
vote
2answers
51 views

Is the set of all projection matrices a convex set?

The set $\phi=\{P| P^2=P\}$ contains all projection matrix. Is this set $\phi$ convex?
0
votes
0answers
41 views

Describing the minimizers of this function

Consider the following continuous function over $x$, with $a,b>0$: $$f(x)=\begin{cases} ax-\sqrt{x} & \text{for }x\leq b^{2}\\ ab\sqrt{x}-b & \text{for }x>b^{2} \end{cases}$$Note that ...
1
vote
0answers
164 views

Proof of Simplex Method, Adjacent CPF Solutions

I was looking at justification as to why the simplex method runs and the basic arguments seem to rely on the follow: i)The optimal solution occurs at some vertex of the feasible region (CPF points) ...
2
votes
2answers
108 views

Conditional expectation of a random vector taking values in convex sets

on a probability space $(\Omega, \mathcal{F},\mathbb{P})$ i have a random vector $X\in L^1_{\mathbb{P}}(\mathbb{R}^d)$ (integrable with values in $\mathbb{R}^n$), such that $\mathcal{P}-a.s.$ $$X\in ...
1
vote
1answer
71 views

Minimization over two lines

This is a minimization question where the minimizing points can be chosen freely on two lines: $$\mbox{minimize}\, \prod_{i=1}^K {y_i}\quad \mbox{such that}\quad \prod_{i=1}^K ...
2
votes
1answer
36 views

Relation between Symmetric matrix and Convex Cone

The "Positive Semedefinite Cone" is defined as $\mathbb{S}^{n}_{+} = \{\mathbf{X}\in\mathbb{S}^{n}: \mathbf{X}\succeq\mathbf{0}\}$. To my knowledge, this representation contains 2 contents: ...
4
votes
1answer
132 views

recover primal solution from dual for matrix completion

Consider the following primal/dual SDPs $$ \min\limits_X \; \lVert X \rVert_* : \mathcal{A}(X) = b \qquad \max\limits_z \; b^T z : \lVert \mathcal{A}^*(z) \rVert \leq 1 $$ where $\lVert X \rVert_* = ...
0
votes
0answers
48 views

$f:D\subset \Bbb R^2 \rightarrow \Bbb R$, where $D$ is a compact and convex set, reaches it maximum at $int(D)$

I'm trying to prove that if $D$ is a compact and convex (for every two elements of $D$, the line that connects them is contained in $D$) then: If $f:D\subset \Bbb R^2 \rightarrow \Bbb R$ and at ...