Convex Optimization is a special case of mathematical optimization. It includes Linear Programming and least-squares.

learn more… | top users | synonyms

2
votes
1answer
42 views

Solve the closed form solution for argmax of $ x^Ty - x^T\ln(x)$

Let $y \in \mathbb{R}^n, \ln(x) = \begin{bmatrix} \ln(x_1) \\ \vdots \\ \ln(x_n) \end{bmatrix} \in \mathbb{R}^n$ Show that $$x^* = \text{argmax}_{x \in \mathcal{D}} \quad x^Ty - x^T\ln(x)$$ ...
0
votes
0answers
17 views

Solving $I^* = \arg\min_{I'} \left( \|\phi_\ell(I) - \phi_\ell(I')\|_2^2 + R(I') \right)$ with gradient descent

I am trying to create the results from this a paper that is trying to understand the types of features a convolutional neural network is learning to recognize. I don't think understanding ...
1
vote
1answer
24 views

Benders decomposition Master Problem

I am currently working on implementation of Bender's Decomposition for MIP. I am looking at the simplest model \begin{equation} \begin{split} \min_{x,y} &\; c^Tx + f(y)\\ s.t. & \; Ax + Dy \ge ...
1
vote
0answers
24 views

Interpolating polynomial such that it is convex in specified region

The problem I have is that I have data at two points $x_1,x_2$ and $x_2>x_1>0$. At these two points, I know that the function $f$ has values $f(x_1)$ and $f(x_2)$ respectively. It is also ...
0
votes
1answer
10 views

Strongly monotone and cocoercive

A map $f:\mathbb{R}^n\rightarrow\mathbb{R}^n$ is $m$-strongly monotone if $$ (x-y)^{\sf T}((f(x)-f(y)) \geq m \|x-y\|_2^2 $$ for $m > 0$ and is $\delta$-cocoercive if $$ (x-y)^{\sf T}((f(x)-f(y)) \...
1
vote
0answers
17 views

Recommend a optimization book with more coding examples?

I am interested in continuous optimization problems. However, I feel it is very difficult for me to understand the classic books such as Convex Optimization or Numerical Optmization. My problem with ...
0
votes
1answer
46 views

maximize 3-variable linear function [version 1.0]

This problem came up when I was trying to solve a bigger, probabilistic problem. So at the end it boils down to this: how can we maximize the function $f(x_2,x_3,x_4) = \frac{18}{100}x_2 + \frac{...
1
vote
0answers
23 views

Gradient Descent and Scale of Data and Objective Function

One way to tune step size in gradient descent is via backtracking line search. backtracking line search (with parameters α ∈ (0, 1/2), β ∈ (0, 1)) starting at $t = 1$, repeat $t := \beta t$ ...
0
votes
0answers
27 views

Sum/product of two functions of two variables are to be minimized

I have two functions $f(x,y)$ and $g(x,y)$ whose sum/product (whichever is possible) is to be minimized. The values of $x,y$ can vary in the interval $0<x,y<1$ (hence none of them can have a ...
1
vote
1answer
24 views

Why $f:R^2\to R$, with $\mbox{dom} f=R^2_+$ and $f(x_1,x_2) =x_1x_2$ is quasiconcave?

Why $f:R^2\to R$, with $\mbox{dom} f=R^2_+$ and $f(x_1,x_2) =x_1x_2$ is quasiconcave? I have tried to use Jensen eniquality to check that superlevel set $\{x\in R^2_+ | x_1x_2 \ge \alpha\}$ is convex....
1
vote
1answer
23 views

Is the root of a sum of squared differences convex?

Let $x \in \mathbb{R}^n$. Let there be a collection of functions $d_i = (x_j - x_k)^2$ (note that the subscripts $j$ and $k$ are fixed for each $d_i$, and there can be repeated use of subscripts on ...
0
votes
1answer
36 views

Related to Caratheodary theorem

If $P$ is a set of vectors $\textbf{x}_i$'s where every $\textbf{x}_i$ is of dimension $d$ and $|P|=K$. In this case at many places I have seen that the vectors $\textbf{x}_2-\textbf{x}_1,\textbf{x}...
0
votes
0answers
11 views

McCormick Envelopes with more then 2 variable

Suppose that I have the function $z=\sin(x)\cdot y$, where $0\leq x<=0.5$ and $0\leq y\leq 1.0$. If I do the best convex relaxation of $\sin$ and than compute $z=\operatorname{Relax} \sin(x) \cdot ...
0
votes
1answer
36 views

What are spurious local optima?

I keep seeing that word "spurious" (when used in the context of optimization), but I'm having trouble finding a good reference on what the definition of the term is.
0
votes
1answer
21 views

floor/ceiling/round functions in the constraints of an optimization?

I have a constrained optimization problem in which I have to impose a "floor" or "ceiling" constraint to the solution. In fact I decided to use these nonlinear rounding functions because I needed to ...
1
vote
0answers
15 views

Demonstrating convexity of a convex optimization problem

I am working on the following problem. Consider the following function $\textit{f}: \mathbb{R^n}$ × $\mathbb{R^n}$ → $\mathbb{R}$. $$f(\vec{z},\vec{d}) := \min_{t \in \mathbb{R},\vec{v} \in \mathbb{...
1
vote
1answer
30 views

On accelerated Proximal Gradient Methods

I am working on accelerated optimization scheme, which unified in the paper by Paul Tseng, "On Accelerated Proximal Gradient Methods for Convex-Concave Optimization". But unfortunately, it is ...
1
vote
1answer
26 views

Is convexity the most general dividing line between “easy” and “hard” optimization problems?

Just got started with Boyd's Convex Optimization. It's great stuff and I see how it directly subsumes the all-important linear programming class of models. However, it seems that if a problem is non-...
0
votes
1answer
14 views

Does projected gradint descent(pgd) results in the same minimizer as the one given by unconstrained gd and projected back on the constrained set?

For $f: \mathbb{R}^n \mapsto \mathbb{R}$ with $f(x) < \infty,\;\forall x \in \mathbb{R}^n$ and for convenience let's assume $f$ is continuously differentiable. Suppose we are trying to solve the ...
4
votes
1answer
54 views

How can I experiment with Lagrange multiplier in QCQP?

Suppose we want to solve following optimization problem (it is a PCA problem in this post) $$ \underset{\mathbf w}{\text{maximize}}~~ \mathbf w^\top \mathbf{Cw} \\ \text{s.t.}~~~~~~ \mathbf w^\top \...
1
vote
1answer
50 views

A question about Lagrange multiplier in optimization

I read @amoeba 's answer in this post, PCA optimization problem is $$ \underset{\mathbf w}{\text{maximize}}~~ \mathbf w^\top \mathbf{Cw} \\ \text{s.t.}~~~~~~ \|\mathbf w\|_2=1 $$ where $\mathbf C$ ...
18
votes
5answers
549 views

How do you prove that $\{ Ax \mid x \geq 0 \}$ is closed?

Let $A$ be a real $m \times n$ matrix. How do you prove that $\{ Ax \mid x \geq 0, x \in \mathbb R^n \}$ is closed (as in, contains all its limit points)? The inequality $x \geq 0$ is ...
0
votes
0answers
27 views

How do I prove that this function is concave on $f_{ij}(x)$?

I am trying to apply convex optimization to the following problem- ${f^*}(x) = \mathop {\arg \max }\limits_{{f_i}(x)} \sum\limits_i {\ln \left\{ {u_i^* - \sum\limits_j {\frac{1}{{\left( {1 - {\rho _{...
0
votes
1answer
48 views

Minimizing a strictly convex function with inequality constraint

So we've been learning about the Kuhn Tucker conditions in my non-linear optimization course and I've been having trouble with this problem: QUestion: description here Question: a strictly convex ...
0
votes
1answer
53 views

Directional Derivative defines Descent Direction

Let $f:\mathbb{R}^m \mapsto \mathbb{R}$ be a proper convex function that is not necessarily differentiable and let $x\in\mathbb{R}^n$ be such that $\mathbf{0} \notin \partial f(x)$. I want to prove ...
2
votes
1answer
28 views

What are some applications of entropy maximization and minimum volume covering ellipsoid?

I'm reading Boyd's Convex Optimization textbook. In particular, I'm currently focusing on Chapter 5 (Duality). There is a frequent recurrence of two examples: Minimum volume covering ellipsoid \...
1
vote
1answer
19 views

Individually checking constraints for convexity in Optimisation problem valid?

I have a quadratic minimisation problem where both the objective fn and constraints have some quadratic terms. (Such as a throttle variable (continous) * On/Off (integer variable)). My question is: ...
0
votes
1answer
27 views

Robusness of median

If we let $X$ be a set of pints in $\mathrm{R}^2$, and let $g(X) = \arg \min_{y \in \mathrm{R}^2} \sum_{x_i \in X} \parallel x_i -y \parallel_2$ (geometric median of $X$). If $X$ and $X'$ are ...
0
votes
0answers
14 views

Uniqueness of projection implies convexity [duplicate]

Prove that for a compact set A in finite dimensional Euclidean space X, A is convex if and only if for any point x in X, the projection of x to A is unique. If we know A is convex, we can show the ...
0
votes
1answer
36 views

Convex or non-convex function

I want to minimize the following function $$\frac{a}{bxy+cd}e^{\frac{a}{bxy+cd}}H+2-\Gamma(1,\frac{eaf}{b(1-x)},\frac{eagf}{bx(1-y)})$$ where $a,b,c,d,e,f,g,H$ are constants and greater than $0$. $\...
0
votes
0answers
13 views

Optimal Transport of a Translated Density

I've been writing an implementation to the Monge-Kantorovich optimal transport problem in Euclidean space with a quadratic cost function. The outline is that we have densities $f(x)$ and $g(y)$ s.t. $...
0
votes
3answers
74 views

Orthogonal Projection onto the $ {\ell}_{\infty} $ Ball

What is the Ortohogonal Projection onto the $ {\ell}_{\infty} $ Ball? Namely, given $ x \in {\mathbb{R}}^{n} $ what would be: $$ {\mathcal{P}}_{ { \left\| \cdot \right\| }_{\infty} \leq 1 } \left( x ...
2
votes
1answer
26 views

Finding minimizer from different order

Let a nonnegative function $f(x,y)$: $\mathbb R^2\to \mathbb R$ be second order continuous differentiable. We also know that $f$ is not convex in its two arguments, but only separately in each of them....
1
vote
0answers
49 views

How to solve a binary generalized assignment problem

I have the following generalized assignment problem: Z=max $\sum_{i=1}^{N}\sum_{j=1}^{M} x_{ij}R_{ij}$ such that $\quad 1)\quad \sum_{j=1}^{M} x_{ij}=1 \quad \forall i$ $\quad\quad\...
1
vote
0answers
52 views

What are some applications of vertex separators?

What are applications of finding a vertex separator that minimizes a cut in a graph. To clarify the problem I am talking about is is given a graph of n vertices and a partition $m_1,m_2,..,m_k $of ...
0
votes
1answer
26 views

Exact line search in convex optimization

For a convex function $f$ what do we know about convexity of the exact line search problem? $$\min_{\alpha \ge 0} f(x+ \alpha p_k)$$ I think because the function is convex and is linear in variable, ...
0
votes
1answer
97 views

Why is the dual cone of $l^1$ is $l^\infty$?

I just noticed somewhere in Convex Optimization that the dual cone of $l^1$ is $l^\infty$! (A diamond in $\mathbb{R}^2$ for $l^1$ is a square in $\mathbb{R}^2$ for $l^\infty$.) In fact I cannot ...
0
votes
0answers
12 views

Minimizing component-wise convex functions

I want to minimize a function $f(\vec x,\vec y)$, whereby $\vec x$ and $\vec y$ are vectors. If I hold $\vec x$ constant, $f(\vec x,\vec y)$ is convex with respect to $\vec y$, and the reverse is true ...
1
vote
1answer
27 views

Showing that this set satisfies the closed criterion

Suppose we have the subset $S = \{ \lambda v \mid \lambda \geq 0 \} + K $, where $v$ is a vector in $\mathbb{R^3}$ and $K$ is a convex hull of six other vectors. How do I show that it satisfies the ...
0
votes
0answers
28 views

Non convex constraint and possible relaxations.

I am trying to solve a minimization problem of the form $\min_{\alpha,u} \|Xu-\alpha\|^{2}_{2}+\lambda\sum_{i,j}^{n}|\alpha_i-\alpha_j|$ subject to $\|u\|_{2}=1$ where $X$ is $n\times p$ and $u$ is $...
0
votes
0answers
28 views

Is generalized mean convex / concave?

The generalized mean can be given using the following equation: $ M_p(x_1, \dots, x_n) = (\frac{1}{n}\sum_i x_i^p)^{1/p} $ Is it convex /concave when $p<1$ ?
0
votes
1answer
26 views

Conjugate of difference of convex functions

I am reading through this tutorial on DC programming and the author makes a startling claim without proof: If $g$ and $h$ are two lower semi-continuous convex function, then the conjugate function of ...
0
votes
0answers
12 views

show that $f(w) = \sum_{i=1}^N (\frac{1}{1+e^{-w^Tx_i}} - t_i)$ is not convex

I am studying logistic regression and in my book it says that using Hessian we can show that $f(w) = \sum_{i=1}^N (\frac{1}{1+e^{-w^Tx_i}} - t_i)$ is not convex. Both $x$ and $t$ are N-Vectors, and $...
1
vote
1answer
25 views

Connection between complementarity problem and optimization problem?

I do not understand the connection between complementarity problems and optimization problems. I have tried to look at other definitions for complementarity problem to see if that would help me with ...
1
vote
2answers
25 views

Variational Inequalities - What excatly does the definition say? Why are they useful?

I am having issues understanding the definition of variational inequalities. We have the following definition: Given a set $X \subset \mathcal{R}^n$ and a mapping $F: X \rightarrow \mathcal{R}^n$ a ...
1
vote
2answers
100 views

Decrease in the size of gradient in gradient descent

Gradient descent reduces the value of the objective function in each iteration. This is repeated until convergence happens. The question is if the norm of gradient has to decrease as well in every ...
2
votes
1answer
38 views

Minimizing the Frobenius norm with linear inequality constraints

How to solve the following system for $\mathbf{C}$ and $\mathbf{a}$: $\min\|\mathbf{X-XC} \,\mbox{diag} (\mathbf{a})\|_F^2$ subject to $\mathbf{c}_{ik}\geq 0$, $1^T \mathbf{c}_k = 1$ and $1-\delta\...
2
votes
2answers
43 views

Solving Optimization Problem (Orthogonal Projection) Using Projected Sub Gradient / Dual Projected Subgradient

Given the following optimization problem (Orthogonal Projection): $$ {\mathcal{P}}_{\mathcal{T}} \left( x \right) = \arg \min _{y \in \mathcal{T} } \left\{ \frac{1}{2} {\left\| x - y \right\|}^{2} \...
0
votes
1answer
47 views

Why do we minimize the squared norm instead of the norm in this optimization problem?

When reading about the optimization problem for Support Vector Machine in Bishop's book (Pattern Recognition and Machine Learning) he wrote that: The optimization problem then simply requires that ...
1
vote
2answers
55 views

Given a set of vectors and a target vector, find the set of scaling factors that minimizes distance of sum of those vectors from target

I have a set of $n$ starting vectors $\vec i_n$ and a target vector $\vec t$. I have a set of scaling factors $a_n$ for which I can compute the sum $\vec s$: $$ \vec s = \sum_{i=1}^n {a_i \vec i_i} $$...