Convex Optimization is a special case of mathematical optimization. It includes Linear Programming and least-squares.

learn more… | top users | synonyms

1
vote
1answer
156 views

How to show this algorithm on positive semidefinite matrices converges to a global maximum determinant

I'm dealing with an algorithm which is supposed to converge to the maximum determinant of certain positive semidefinite matrices. The problem is that we have such a matrix, and we vary certain ...
0
votes
1answer
118 views

Can a Lipschitz continuous function be strictly convex?

Let $\varphi:\mathbb R^n\to\mathbb R$, and suppose for all $x,y\in\mathbb R^n$, $$\|\varphi(x)-\varphi(y)\|\leq L\|x-y\|$$ for Lipschitz constant $L$. Is it possible for such a function to satisfy ...
1
vote
2answers
137 views

formulating the dual for an instance of a SOCP with linear constraints

I have an optimization problem with second-order cone constraints and linear inequalities and inequalities (shown below). I want to formulate the dual, but have been having trouble. ...
0
votes
2answers
113 views

Symmetric Positive Definite and Gradient Proof

I have the function $f(x)=\frac {1}{2} \mathbf x^T Q \mathbf x - \mathbf b^T \mathbf x$ where $Q$ is symmetric. I'm trying to show that solving $\nabla f(\mathbf x) = 0$ is equivalent to solving $Q ...
1
vote
0answers
34 views

Dual of the mixed $\ell_1/\ell_2$ norm?

The mixed $\ell_1/\ell_2$ norm $\Omega_{12} $ is defined as $\Omega_{12}(x) = \sum_g ||x_g||_2$ where $x_g$ are disjoint subsets of the elements of the vector $x$. This is used in machine learning ...
1
vote
0answers
36 views

can we prove that a certain supremum of affine functions is frechet differentiable or at least continuous?

Let $X$ be a Hilbert space. Let $A\colon \operatorname{dom} A\to X$ be linear operator with closed graph (not necessarily bounded). Define $$ g\colon \operatorname{dom}A\to \mathbb{R}:x\mapsto ...
3
votes
2answers
1k views

Adding Elements to Diagonal of Symmetric Matrix to Ensure Positive Definiteness.

I have a symmetric matrix $A$, which has zeroes all along the diagonal i.e. $A_{ii}=0$. I cannot change the off diagonal elements of this matrix, I can only change the diagonal elements. I need this ...
-1
votes
1answer
123 views

Constrained optimization with complex variables

Is there a theory of constrained optimization with complex variables, do you know any textbook on that topic? The typical textbooks on constrained optimization deal with real variables. I actually ...
1
vote
2answers
81 views

Hessian matrix and epigraphs

I'm working on a homework assignment concerning convex optimization and I came across a problem involving the convexity of the function and the convexity of the domain of the function. Consider the ...
1
vote
1answer
43 views

Optimization of several cost functions together

Say I want to minimize several functions together: $$\min \lVert f_1\rVert, \min \lVert f_2\rVert, \min \lVert f_1-f_2\rVert$$ where $\lVert f\rVert$ is the $L_2$ norm of $f$. I am wondering can I ...
1
vote
1answer
46 views

Convex Function Help and Counterxample

Given $g: \mathbb{R}^n \to \mathbb{R}$ is convex and $f:\mathbb{R} \to \mathbb{R}$ is convex and increasing. Show that $(f \circ g): \mathbb{R}^n \to \mathbb{R}$ is convex. I had no problem proving ...
2
votes
0answers
83 views

How to prove the convexity of the logrithmic gamma function?

Here's what I did: $$\Gamma'(z)=\int_0^\infty \log(t)e^{-t}t^{z-1}dt$$ $$\Gamma''(z)=\int_0^\infty ...
1
vote
1answer
35 views

Approximate an exponential function

I have an optimization problem, where I would like to minimize $$F=\exp(\mathrm{trace}(A)+\frac{1}{2}\mathrm{trace}(A^2)-\lambda)$$ where $A$ is a non-negative matrix. Is it possible to replace $F$ ...
0
votes
1answer
351 views

Composition of convex function and affine function

Let $g: E^{m} \rightarrow E^{1}$ be a convex function, and let $h: E^{n} \rightarrow E^{m} $ be an affine function of the form $h(x)=Ax+b$, where $A$ is an $m \times n$ matrix and $b$ is an $m \times ...
-1
votes
1answer
91 views

Prove or disprove the concavity of the function [closed]

Prove or disprove the concavity of $f$ over the following two domains. $$f(x_1,x_2)=10-2(x_2-x^{2}_{1})^{2}$$ defined either over $$S_1=\{(x_1,x_2) : -1\leq x_1 \leq 1, -1 \leq x_2 \leq ...
-1
votes
2answers
41 views

Constraints in optimization; redundant hardness?

This is not an accurate mathematical problem, and rather a philosophical and ambitious question. As far as I know, unconstrained problems are easier than constrained problems; right? This is mostly ...
1
vote
1answer
358 views

convexity of matrix “soft-max” (log trace of matrix exponential)

In convex optimization it is often convenient to use the following smooth approximation to $\max\{x_1, \ldots, x_n\}$: $$ f_\lambda(x_1, \ldots, x_n) = \frac{1}{\lambda}\log \sum_{i = 1}^n{e^{\lambda ...
2
votes
1answer
894 views

How to find closest positive definite matrix of non-symmetric matrix

I have a matrix A given and I want to find the matrix B which is closest to A in the frobenius norm and is positiv definite. B does not need to be symmetric. I found a lot of solutions if the input ...
1
vote
0answers
21 views

Uniqueness of the solution to a quadratic opt problem

Consider a positive definite matrix $\boldsymbol H$, the known vectors ${\boldsymbol b}$ and ${\boldsymbol a}_i$. Now the minimization problem is casted with respect to the vector ${\boldsymbol x} $ ...
8
votes
1answer
306 views

Stochastic gradient descent for convex optimization

What happens if a convex objective is optimized by stochastic gradient descent? Is a global solution achieved?
0
votes
2answers
47 views

Show that the set of points that are nearer $a$ than $b$ with respect to $\lVert \cdot \rVert_2$ is convex

I am trying to show the above statement: Show that the set of points that are nearer $a$ than $b$ in the sense of Euclidean $\lVert\cdot\rVert_2$ are convex. My attempt This follows from the ...
2
votes
1answer
166 views

The convex conjugate of a quadratic form with positive semi definite matrix

I want to find the convex conjugate (Legendre transform) of a quadratic for $1/2x^{t}Qx$ when $Q$ is positive semi-definite. If Q is non-singular, the solution is easy - the gradient is $y-Qx=0$ so ...
2
votes
2answers
290 views

What change of variables, if any, transforms this nonconvex problem into a convex one?

I'm looking for a convex reformulation, if any exists, of the following minimisation problem: Let $A$ be a symmetric, positive definite $n \times n$ matrix, and $b \in \mathbb{R}^n$. Minimise ...
0
votes
1answer
81 views

Cyclic monotonicity of sub-differential domain and convex property

I am looking for hints/proof's overview/reference about this proposition : Let $S\subset \mathbb{R}^d\times\mathbb{R}^d$. There exist a convex function $\phi$ such that $S\subset \partial\phi$ ...
2
votes
3answers
124 views

Is this set convex ?2

Is this set convex for every arbitrary $\alpha\in \mathbb R$? $$\Big\{(x_1,x_2)\in \mathbb R^2_{++} \,\Big|\, x_1x_2\geq \alpha\Big\}$$ Where $\mathbb R^2_{++}=[0,+\infty)\times [0,+\infty)$.
2
votes
0answers
55 views

Strictly convex self-concordant function

Some definitions: A function $f:R^n\rightarrow R$ is convex[strictly convex] if for every $\lambda\in[0,1]$ [$\lambda\in(0,1)$] and for every $x,y$ [$x\neq y$] in $R^n$ we have $f(\lambda ...
1
vote
0answers
271 views

Minimum L1 norm may not obtain the sparsest solution?

Here is a paper called For Most Large Underdetermined Systems of Equations, the Minimal L1-norm Near-Solution Approximates the Sparsest Near-Solution. However, I did not quite get its definition of ...
1
vote
0answers
79 views

Integral Farkas Lemma

The context of this question is commutative algebra, however the question itself is more related to convex geometry. All necessary information is given. In the proof of Lemma 3.1.1 in the book ...
1
vote
1answer
135 views

minimum trace norm on the set of matrices with fixed diagonal entries

What is the min nuclear norm (sum of singular values) on all $n \times n$ matrices$A$ whose diagonal is fixed. i.e. $diag(A) = v$ Is it true that the diagonal matrix is a minimizer?
1
vote
1answer
44 views

Minimise Total $x$, Maximum $x$ or $|x|$ in integer/linear programming

Suppose we have a linear program (it may be integer, it probably doesn't matter). Suppose the variables $t_j$ give the tardiness for job $j$, and we want to minimise something to do with this ...
2
votes
2answers
108 views

Verifying the convexity of some function

Convex function: We will say that $f:X\rightarrow R$ is convex function if for every $\lambda\in [0,1]$ and for every $x,y\in X$ ($X$ is convex space) $f(\lambda x+(1-\lambda)y)\leq\lambda ...
0
votes
1answer
47 views

Proximal mapping scaling property

According to this presentation it holds that for $h(x) = f(\lambda x)$ it holds that $prox_h(x) = \frac{1}{\lambda} prox_{\lambda^2 f}(\lambda x)$ where the proximal operator is defined as ...
0
votes
1answer
81 views

Strictly positive solution of linear equations

Suppose $A\in\mathbb{R}^{m\times n}$, $b\in\mathbb{R}^m$, and $b\in \mathcal{R}(A)$. Show that there exists an $x$ satisfying $x \succcurlyeq 0$, $Ax = b$ if and only if there exists no ...
1
vote
1answer
143 views

Free software or algorithm for Second-Order Cone Program

I need to solve the following optimization problem: $$ \mathbf{x}^\ast = \operatorname{argmin}_{\mathbf{x}} \Vert \mathbf{Rx} \Vert_2^2 \;\;\; \mathrm{s.t.} \;\;\; \mathbf{s}^\mathrm{H} \mathbf{x} = ...
3
votes
1answer
169 views

Lagrange dual of a sum of convex functions

Given a set of convex functions $f_i(x)$ and convex sets $X_i$ in $\mathbb R^n$ I need to find the Lagrange dual problem for the problem $\min \sum{f_i(x)} , x \in X_i \forall i$. There is of course ...
2
votes
2answers
310 views

Why the unit circle in $\mathbf{R^2}$ has one dimension?

When I was reading 'Convex Optimization, Stephen Boyd', I was wondering of following steps Consider the unit circle in $\mathbf{R^2}$, $i.e.$, $\{x\in\mathbf{R^2}|x^2_1+x^2_2=1\}$. Its affine hull ...
0
votes
1answer
127 views

Lagrangian dual for the sum of norms

I would like some help in deriving the Lagrangian dual function of a sum-of-norms minimization problem : $\sum{||A_{i}x-b_{i}||}$ when $A_{i}$ are matrices, and $b_{i},x$ vectors. I understand I can ...
0
votes
2answers
91 views

machine learning optimization

I was studying SVM and I am having problems in the conversion of this optimization problem into another : and gamma_hat is defined by I had to paste the images because I was having troubles with ...
3
votes
2answers
412 views

Recovering the solution of optimization problem from the dual problem

In the context of (most of the times convex) optimization problems - I understand that I can build a Lagrange dual problem and assuming I know there is strong duality (no gap) I can find the optimum ...
0
votes
1answer
55 views

Finding the Expansion of a Separable Convex Optimization Problem

Hi there is a convex optimization problem in this paper which I am trying to implement in mosek. The author specifies that they also implement it using the separable optimization method. Specifically ...
0
votes
1answer
62 views

SOCP formulation: wrong inequality direction in constraints

The problem is constrained by a set of inequalities in the form of $$ \| A_i\mathbf{x}\|\geq \mathbf{y_i^Tx} $$ where x is a n-vector of unknowns, $A_i$ are matrices and $y_i$ vectors. Is it possible ...
0
votes
1answer
64 views

Probability that a $n$-dimensional Gaussian falls into a half-space

For $a \in \mathbb{R}_{\ge 0}^d$ and $b \in \mathbb{R}_{\ge 0}$, we can define a half-space as the set of points $x \in \mathbb{R}^d$ such that $a \cdot x \le b$, namely, $$\mathcal{H}(a,b) = \{x \in ...
1
vote
2answers
93 views

Trace of quadratic function with 2 PSD matrices - convex?

If A & B are positive semi-definite, is this always convex: $$ trace(XAX^TB) $$ There was a similar question asked here: Trace of a quadratic function, Convexity and here: Confusion related to ...
2
votes
1answer
102 views

Convexity of a region on probability simplex

Exercise 2.15 g of Boyd et al Convex Optimization book : On the probability simplex in $\mathbb{R}^n$ where each point $p = (p_1,p_2,p_3,\ldots,p_n)$ corresponds to a distribution for random variable ...
1
vote
1answer
145 views

how to construct the Lagrangian dual problem?

The primal optimization problem is, \begin{align*}\min_x\;&f_0(x)\\ \text{s.t.}\;&f_i(x)\le0\\ &h_j(x)=0\end{align*}, to construct the dual problem, I form the Lagrangian, ...
2
votes
1answer
52 views

What is the name of this object?

Suppose I have a convex set $K\subset X$, where $X$ is say a real Hilbert space (for simplicity). Then, given some $a\in \Bbb{R}$, let $$ \hat{K}=\{x:\langle x,y\rangle \leq a \;\forall y\in K\} $$ ...
1
vote
2answers
127 views

What is the motivation behind strong convexity

Definition : A function is said to be $\beta$-strongly convex if, $f(\theta w + (1-\theta) w') \le \theta f(w) + (1-\theta) f(w') - \frac{\beta}{2}\theta(1-\theta)(w-w')^2$ What is the motivation ...
4
votes
3answers
264 views

Newton's method intuition

In optimisation the Newton step is $-\nabla^2f(x)^{-1}\nabla f(x)$. Could someone offer an intuitive explanation of why the Newton direction is a good search direction? For example I can think of ...
4
votes
1answer
642 views

What is the time complexity of conjugate gradient method

I have been trying to figure our the time complexity of conjugate gradient method I have to solve a system of linear equations given by $$ Ax=b $$ where A is sparse and positive definite symmetrix ...
0
votes
1answer
54 views

Prove that proximal function is convex

How to prove that the proximal function $$ \Phi (y) \equiv \min_x \left(f(x)+\frac{1}{2} ||x-y||_2^2\right) $$ is a convex function of $y$ if $f(x)$ is a convex function of $x\in \mathbb{R}^n $? ...