Convex Optimization is a special case of mathematical optimization. It includes Linear Programming and least-squares.

learn more… | top users | synonyms

35
votes
4answers
4k views

Please explain the intuition behind the dual problem in optimization.

I've studied convex optimization pretty carefully, but don't feel that I have yet "grokked" the dual problem. Here are some questions I would like to understand more deeply/clearly/simply: 1) How ...
17
votes
1answer
656 views

Properties of the Cone of Positive Semidefinite Matrices

The set of positive semidefinite symmetric real matrices form a cone. We can define an order over the set of matrices by saying $X\geq Y$ if and only if $X-Y$ is positive semidefinite. I suspect that ...
15
votes
4answers
8k views

Difference between supremum and maximum

Referring to this lecture , I want to know what is the difference between supremum and maximum. It looks same as far as the lecture is concerned when it explains pointwise supremum and pointwise ...
11
votes
1answer
449 views

invex functions and their usefulness?

An invex function $f$ is a differentiable function from $\Bbb R^n$ to $\Bbb R$ that for some function $\eta : \Bbb R^n \times \Bbb R^n \to \Bbb R^n$ satisfies for all $x, u$, $f(x) - f(u) \geq \eta(x, ...
10
votes
2answers
647 views

If the Minkowski sum of two convex closed sets is a Euclidean ball, then can the two sets be anything other than Euclidean balls?

If for two convex closed sets $S_1$ and $S_2$, the Minkowski sum is a Euclidean ball then can $S_1$ and $S_2$ be anything other than Euclidean balls themselves. I suspect they can be but I haven't ...
10
votes
2answers
2k views

What is the difference between minimum and infimum?

What is the difference between minimum and infimum? I have a great confusion about this.
10
votes
5answers
936 views

Why is convexity more important than quasi-convexity in optimization?

In the mathematical optimization literature it is common to distinguish problems according to whether or not they are convex. The reason seems to be that convex problems are guaranteed to have ...
10
votes
1answer
385 views

A (mathematically) sound investment strategy

It is common wisdom in the investment community that a long-term investor saving for his future would do well to invest in high-risk/high-return assets when he is young, slowly switching his portfolio ...
9
votes
5answers
876 views

Derivative of nuclear norm

I'm trying to take the derivative of nuclear norm with respect to its argument. nuclear norm is defined in the following way: $$\|x\|_*=\mathrm{tr}(\sqrt{x^Tx})$$ I'm trying to calculate: ...
9
votes
2answers
349 views

On the convexity of element-wise norm 1 of the inverse

Let us define $\|A\|_1$ the element wise norm 1 of a matrix $A \in \mathbb{R}^{n \times m}$ as $$ \|A\|_1= \sum_{i,j} |A_{i,j}|. $$ Obviously, this function is convex over $\mathbb{R}^{n \times m}$. ...
8
votes
1answer
248 views

Stochastic gradient descent for convex optimization

What happens if a convex objective is optimized by stochastic gradient descent? Is a global solution achieved?
8
votes
1answer
5k views

L1 norm and L2 norm

I was studying the Stephen Boyd's textbook on convex optimization. It says the following: The amplitude distribution of the optimal residual for the l1-norm approximation problem will tend to have ...
7
votes
3answers
131 views

Is group theory useful in any way to optimization?

For what I have seen, optimization uses a lot of linear algebra and convex analysis, but I have not seen any group theory being used, so I was curious about it. Is group theory useful in any way to ...
7
votes
4answers
839 views

Why is the affine hull of the unit circle $\mathbb R^2$?

In Boyd's "Convex Optimization" it defines the affine hull of a subset $C$ of $\mathbb R^n$ as $$\text{aff} C = \left\{\theta_1 x_1 + \ldots +\theta_k x_k \mid x_1, \ldots x_k \in C, \theta_1 + ...
7
votes
1answer
259 views

How to understand convex duality intuitively

Is there an intuitive way to understand the convex duality? If the primal problem is minimization, the dual is maximization over another set of variables - but I would love to have a geometric ...
6
votes
3answers
1k views

Convexity and Affineness

In reading about convex optimization, the author states that all convex sets are affine. Are affinity and convexity equivalent? If I understand, both definitions incorporate the notion that a set is ...
6
votes
1answer
108 views

Can we test whether a polynomial only takes non-negative values on the non-negative orthant?

EDIT: Feel free to replace "non-negative on the non-negative orthant" with "non-negative on a convex set, cone, or any other class of sets that includes the orthant". A popular way to establish that ...
6
votes
1answer
162 views

Intuition behind accelerated first-order methods

$\newcommand{\prox}{\operatorname{prox}}$ $\newcommand{\argmin}{\operatorname{argmin}}$ Suppose that we want to solve the following convex optimization problem: $\min_{x \in \mathbb{R}^n} g(x) + ...
6
votes
0answers
294 views

Is this question solvable? $2$ non-linear equations and the proof that the solution is unique (with asymmetric bounty option)

As mentioned in the title I want to show the uniqueness of the solution to $2$ non-linear equations. However, it seems that I can not solve this question with my current mathematical knowledge. More ...
6
votes
0answers
192 views

Subgradient of convex minimization duality

$$\min(f_0(x))$$ $$\text{s.t. }f_i(x) \le y_i \forall i, i = 1 ,\ldots, m$$ $$f_i : \text{convex};\quad x : \text{variable}$$ It is also considered that $g(y)$ is the optimal value of the problem ...
6
votes
0answers
183 views

Optimization of relative entropy

Wondering if my following question is an application of information theory: Lets say we have a factory and ship boxes of stuff outside. If a competitor stands outside my factory, observes the stream ...
5
votes
1answer
4k views

Convexity of the product of two functions in higher dimensions

Exercise 3.32 page 119 of Convex Optimization is concerned with the proof that if $f:\mathbb{R}\rightarrow\mathbb{R}:x\mapsto f(x)$ and $g:\mathbb{R}\rightarrow\mathbb{R}:x\mapsto g(x)$ are both ...
5
votes
3answers
342 views

Is inverse matrix convex?

I wonder a generalization of Jensen's inequality: let $\mathbf{X,Y}$ be two positive definite matrices, can we obtain the following Jensen like inequality ...
5
votes
1answer
3k views

How the dual LP solves the primal LP

When I heard someone discussing LP the other day, I heard him say, "Well, we could just solve the dual." I know that both the primal LP and its dual must have the same optimal objective value ...
5
votes
3answers
1k views

Easy convex sets question

Question 2.23 out of Boyd and Vanderberghe: Give an example of two closed convex sets that are disjoint but cannot be strictly separated. The obvious idea is to take something like unbounded sets ...
5
votes
1answer
133 views

a conjecture on norms and convex functions over polytopes

Suppose one has a convex, bounded polytope P $\subset R^n$ and a strictly convex function $f$ defined everywhere on $R^n$. $f$ has a unique minimum; and suppose this minimum occurs somewhere strictly ...
5
votes
1answer
6k views

Degeneracy in Linear Programming

Consider the standard form polyhedron, and assume that the rows of the matrix A are linearly independent. $$ \left \{ x | Ax = b, x \geq 0 \right \} $$ (a) Suppose that two different bases lead to ...
5
votes
1answer
165 views

Is this function concave or can it be made concave?

I am working with a point process with an event arrival rate of: $$ \lambda(t) = \mu + \sum\limits_{t_i<t}{\alpha e^{-\beta(t-t_i)}}$$ where $ t_1,..t_n $ are the event arrival times. The log ...
5
votes
1answer
75 views

Constrained Optimizatoin: The Frank-Wolfe Method

A general convex optimization problem is framed as such: $$\min f(x) : x \in \Omega$$ where $\Omega$ is convex. The Frank-Wolfe method seeks a feasible descent direction $d_k$ (i.e. $x_k + d_k \in ...
5
votes
1answer
578 views

Solving a set of 3 nonlinear equations with constraints

Problem statement: I am given 3 sets of equations that govern the force $P$, and also the neutral axis, defined by two variables, the radius from the center $r$ and also the rotation degree in ...
5
votes
1answer
427 views

Dual to the dual norm is the original norm (?)

I have the following questions about dual norms : How do you prove that the dual of the dual norm is in fact the original norm? This is what I have so far: If I have $\|y\|_* $ as the norm dual of ...
5
votes
1answer
1k views

Convexity of log det X???

In Boyd's book on convex optimization he proves convexity of log det X by proving it to be concave along a line i.e. he proves that the Hessian of the function $g(t) = f(Z+tV)$ is negative therefore ...
5
votes
1answer
563 views

Minimizing l-infinity norm of complex vector

I have an $n$-dimensional complex vector space, and I want to minimize the $L_\infty$ norm of a point that is constrained to an $m$-dimensional affine subspace. That is, Given $\mathbf{z} \in ...
5
votes
0answers
160 views

Need advice: what should be my next step?

I am dealing with a quite algebraic question and I arrived at some good point. I had $2$ equations with $2$ unknowns and I was able to eleminate one of the variables. My final equation still seems ...
4
votes
2answers
180 views

Is the rising factorial function a convex function?

Let $(x)_p=x(x+1)\dots(x+p-1)$ be the rising factorial function. My question is: Is $(x)_p$ a convex function or not? And how to proof? And what is about the falling factorial function ...
4
votes
4answers
642 views

Summary of Optimization Methods.

Context: So in a lot of my self-studies, I come across ways to solve problems that involve optimization of some objective function. (I am coming from signal processing background). Anyway, I seem to ...
4
votes
2answers
122 views

proving this inequality related to conjugate functions

For $x \in \mathbb{R}^n$ let us denote $x_{[i]}$ the $i$th largest component of $x$ s.t $$ x_{[1]} \geq x_{[2]} \geq x_{[3]}\ge\cdots $$ The function $$ f(x)= \sum_{i=1}^r x_{[i]} $$ is the sum of ...
4
votes
1answer
2k views

prove this is a strongly convex function

The definition of strongly convex from Wikipedia: It is not necessary for a function to be differentiable in order to be strongly convex. A third definition for a strongly convex function, with ...
4
votes
1answer
83 views

Are all non-convex problems created equal?

The distinction between convex and non-convex problems is usually dubbed as the distinction between easy and hard problems. While in the convex case you are golden (local optima are global optima; ...
4
votes
3answers
203 views

Newton's method intuition

In optimisation the Newton step is $-\nabla^2f(x)^{-1}\nabla f(x)$. Could someone offer an intuitive explanation of why the Newton direction is a good search direction? For example I can think of ...
4
votes
1answer
467 views

Both convex and concave functions

Let $f$ be a function from $\mathbb{R}^n$ to $\mathbb{R}$, which is convex & concave and continuous with $f(0)=0$. How to prove that $f(x)=q\cdot x$ for all $x$ in $\mathbb{R}^n$, for a scalar ...
4
votes
1answer
1k views

How to prove this function is concave?

This is the function: $\displaystyle f(\vec x) = \log \frac{\exp(x_1)}{\sum_{i=1}^n \exp(x_i)} $
4
votes
2answers
262 views

Optimization problem using Reproducing Kernel Hilbert Spaces

I am encountering a problem concerning Reproducing Kernel Hilbert Spaces (RKHS) in the context of machine learning using Support Vector Machines (SVMs). With refernce to this paper [Olivier Chapelle, ...
4
votes
1answer
568 views

Help me organize these concepts — KKT conditions and dual problem

This is a long question in which I explain my current understanding of certain ideas. If anyone is interested in reading this and would like to provide any commentary/feedback that may help me ...
4
votes
1answer
232 views

$L_1$ projection of sum of convex functions onto polytopes

Suppose I have a function $f(x) : \mathbb R^n \to \mathbb R$ that is the sum of a given strictly convex function $g : \mathbb R \to \mathbb R$ in a single variable, i.e. $f(x) = g(x_1) + g(x_2) + ...
4
votes
1answer
327 views

Projection of a matrix onto the spectral norm ball?

I'm trying to solve this minimization problem: given a matrix $X$ with at least one singular value greater than 1, find $ \min_Z\; \langle X-Z,X-Z \rangle$ subject to the constraint $ \|Z\| = ...
4
votes
1answer
474 views

Question about the simplex method complexity

So I know that in general the simplex method for linear and convex quadratic programming can require exponential time. But assuming a positive semidefinite quadratic program that is solvable by the ...
4
votes
1answer
219 views

A conjecture of parallelogram inside convex and central symmetric curve

Assume Q is a convex central symmetric curve, whose area is $\displaystyle S$. The area of the maximum parallelogram inside Q is $\displaystyle S'$. How to prove the conjecture that $\displaystyle ...
4
votes
1answer
1k views

Proof of Hoeffding's inequality

The Hoeffding's inequality is $P(S_n - E[S_n] \geq \epsilon) \leq e^{-2\epsilon^2/k'}$, where $S_n = \sum_{i=1}^{n} X_i$, $X_i$'s are independent bounded random variables, and $k'$ depends on the ...
4
votes
2answers
380 views

Conjugates of norms

How would one find the conjugate of the following : $$f(x) = \|x\|^2 /2$$ The conjugate function is defined as $ f^*(y) = \max_x y^Tx - f(x)$ I am stuck at how I can derive the explicit form for ...