1
vote
2answers
42 views

Is the set of all projection matrices a convex set?

The set $\phi=\{P| P^2=P\}$ contains all projection matrix. Is this set $\phi$ convex?
0
votes
0answers
22 views

Conservative perceptron update rule - convex optimization

Suppose I have a condition on a perceptron update rule should be a little conservative. For example, it minimizes the distance between the new update and previous classifier $w_i$, i.e. $||w_{i+1} - ...
0
votes
1answer
64 views

Give example of a set which has No Extreme Point !!..

Give example of a set in R^2 , which has no extreme point ?? We were given this question for assignment !!..I thought of a simple line but doing some research i stumbled upon this solution which ...
2
votes
1answer
29 views

Matrix representation of the following equation - for finding optimal weights for regularized linear regression

If I have the following equation, $$E(w)=\sum_{i=1}^n (y_n -\beta^T x_n) +\lambda \sum_{i=1}^d \beta_i^2 $$ which is the cost function of regularized linear regression ($\beta$ and $x_n$ are ...
0
votes
1answer
36 views

Question about the structure of a convex optimization problem

I am reading a tutorial about Convex Optimization and it defines the general Convex Optimization problem as: $$ minimize_x f(x)$$ where $g_1(x) \leq 0, ... , g_m(x) \leq 0$ and $Ax=b$ and $x \in ...
1
vote
1answer
24 views

Proving an affine set's equivalence to the solution set of $Ax=b$

I am stuck with the following equivalence about Affine Sets: "$L$ being an affine set is equivalent to $L$ being the solution set of a set of equations $Ax=b$ for some $A,b$." In a more mathematical ...
0
votes
1answer
27 views

Basis of the row-space of a matrix with non-negative entries.

Consider a matrix $A \in \mathbb{R}^{n \times m}$ such that all entries are non-negative. Denote the rank of $A$ as $k$. I am mostly interested in cases where $k \ll n$, but this probably isn't ...
0
votes
1answer
34 views

The relation between two different definitions of Affine sets

I am following a presentation, which says that for an affine set $L \subseteq \mathbb{R}^n$ it is: $$L=\left\{x|Ax=b \right\}$$ for some $A,b$. The first definition of $L$ as an affine set is given ...
0
votes
0answers
39 views

How to Extract the dual feasible search directions for the primal-dual potential reduction algorithm?

I am trying to implement the 4.4 Primal-dual potential reduction algorithm introduced in M.S Lobo et al.. Here is a screenshot depicts the algorithm flow: As ...
1
vote
1answer
42 views

Relation between Riccati Algebraic Equation and optimization problem

Reading this page: http://www.mathworks.com/help/robust/ug/minimizing-linear-objectives-under-lmi-constraints.html I got stuck in the result that says it can be show that minimizing Trace of X (a ...
3
votes
0answers
32 views

Easy elementary proof of Farkas Lemma?

Is there any elementary proof of Farkas lemma which does not use convex analysis and hyperplane separation theorem? What about special case below: If the Matrix $A$ is invertible, then there is ...
0
votes
0answers
21 views

A Difficult combinatorial optimization problem

Let $\mathcal{J}$ be a closed, bounded, compact, convex set in $\mathbb{R}^L$. (Notations: vector $\mathbf{x}$ is denoted in bold letters and its $i^{th}$ co-ordinate is denoted as $x_i$. ...
0
votes
0answers
67 views

weights go to infinity in logistic regression with linearly separable data

I have the loss function of logistic regression $L(W)$ = - $\sum_{i=1}^n {y_i}.log[\sigma(w^Tx)] + {(1-y_i)}.log[1- \sigma(w^Tx)]$ I have derived the Hessian and proven it's positive semi-definite ...
0
votes
1answer
54 views

Solving LP with two $L_1$ inequality constraints

Is there a "fast" way to solve the following LP formulation with the following constraints: $$ \max_{\mathbf{f}} \mathbf{f}'.\mathbf{g} \\ \mathbf{1}'\mathbf{f}=1\\ \|\mathbf{f}-\mathbf{h}\|_1\le ...
0
votes
1answer
39 views

Convex matrix function

Please give me some hints for the following problem: Let $S = \{D \in R^{m \times n}, \|d_i\| \leq 1, i = 1, 2, \dots, n \}$. Find condition of $F \in R^{m \times m}$ such that the function: $ f(D) ...
0
votes
1answer
109 views

Closed form solution

I have the following optimization problem: $$\min_{\mathbf{G}} \|\mathbf{B(A+G)\|_F^2} \quad{} \\\text{subject to} \quad{} \mathbf{\|C^T(A+G)\|_F^2\leq \gamma \|A^T(A+G)\|_F^2 } \quad{}, \\ ...
0
votes
1answer
32 views

what is the dual of the following linear program over a convex set?

Let $\mathbf{x}=[x_0,x_1,\dots,x_N]^T$ be a $(N+1)\times 1$vector. Let $\mathcal{S}$ be a bounded, compact convex set in strictly positive quadrant of $\mathbb{R}^{N+1}$. Consider the following ...
0
votes
1answer
31 views

Reentrant constraints in active set algorithm?

Problem definition Supposing you're trying to solve a quadratic program: $$ \min_x f(x) = \frac{1}{2}x^T Q x + c^T x \\ \mbox{s.t} \, \; A x \ge b$$ Where Q is square ($n$x$n$), positive semi ...
1
vote
0answers
21 views

How to convexify (relax) this L0 eigenvalue optimization problem?

Let $C_1,\dots,C_L$ be $N\times N$ hermitian matrices. Let $d<0$ be a given negative constant. Then consider the optimization problem \begin{align} \max_{r\in \mathcal{R}^{L\times 1}} &\mid\mid ...
0
votes
1answer
27 views

Is $f(X) = || Y - XX^T ||_F$ convex given fixed $Y$?

In the scene of nonnegative matrix factorization, $f(X_1, X_2) = \| Y - X_1 X_2 \|_F$ is not convex, but both $f(X_1)$ given fixed $X_2$ and $f(X_2)$ given fixed $X_1$ are convex, enabling us to ...
0
votes
0answers
105 views

Maximize the maximum Eigenvalue under a diagonally constrained matrix

Suppose we have $N\times N$ Hermitian matrix $\mathbf{A}$ I want to find the real $N\times N$ diagonal matrix $\mathbf{D}$ that maximizes the sum of the maximum Eigenvalues : $\mathbf{D}=\arg\max ...
1
vote
1answer
33 views

Problem understanding dual optimization problem?

I am reading this paper: http://dl.acm.org/citation.cfm?id=1390696 Following optimization problem is defined in section 2: \begin{align} \max_{\mathbf{X}>0} \log ...
0
votes
1answer
39 views

Projection onto Polyeder

I know how to projects onto a linear subspace of $\mathbb R^3$, but how to project a point $x$ onto an polyhedron given as the intersection of three halfspaces $$ \langle y_1, x \rangle \ge c_1 ...
3
votes
1answer
77 views

Rank one plus diagonal matrix approximation

Given $A \in R^{n \times n}$, $A$ symmetric. I'm trying to solve the following minimization problem: $\underset{u \in R^n, d \in R^n} \min \, \frac{1}{2} \|X - A\|_F^2$ subject to $X = u u^T + ...
0
votes
0answers
34 views

Union of all sets of optimal solutions to a perturbed linear programming problem

Please let me know if you have some ideas on how to approach this proof? I got stuck part-way through. The following linear program is a function of $\theta$, $ \begin{array}{ll} \min & c^\top x ...
0
votes
1answer
31 views

Is this following SDP problem Convex?

Is the following problem convex function? \begin{eqnarray} \begin{aligned} \underset{\mathbf{X}} {\text{minimize}}\,\,\,\, & \text{Trace}(\mathbf{RX}) \\ \text{subject to} &\\ & ...
0
votes
0answers
22 views

Robust LP question using box uncertainty model

I am trying to solve this robust LP problem by writing it as a QP $$\min_x x^TSx : \mu \leq r^T x , Ax \leq b$$ Under Box uncertainty model: $$R = \{r : \| r - \hat{r}\|_\infty \leq \rho\}$$ Here ...
0
votes
1answer
52 views

Homogeneous non-negative least-squares

I would like to least-squares-"solve" a set of linear equations ($\underset{\mathbf{x}}{\mathrm{argmin}}\; \|\mathbf{Ax-b}\|_2$). In my case, $\mathbf{b=0}$, e.g. the system is homogeneous. I also ...
1
vote
2answers
70 views

Optimization of Product of Different Objective functions (Ex.: Maximize The Product of projections of a complex vector)

Suppose We have this optimization problem which is convex $\mathbf{x}={\arg}\: \underset{\mathbf{x}}\max f_{i}\left (\mathbf{x} \right )$ But the product of different objective function is ...
0
votes
2answers
73 views

Solution in general for a seemingly simple problem

Let $\mathbb{S}$ be a closed, bounded, convex set in $\mathbb{R}^N$. Let $\mathbb{x}=[x_1,\dots,x_N]$ be any arbitrary vector in $\mathbb{S}$. Then what can we comment on the problem \begin{align} ...
0
votes
0answers
55 views

Expressing rank condition of a matrix in terms of its elements

Let $x \in \mathbb{R}^{n}$, define $X = xx^{T}$. I have an optimization problem with some linear constraints and few quadratic constraints, and I have to solve for $x$. Using $X$ as the unknown ...
0
votes
1answer
588 views

Semi-positive definite Hessian matrix and local minimum

Suppose we have a function $F(x)$ defined as \begin{equation} F(x) = \frac{1}{2}x^TAx + b^Tx +c, \end{equation} where \begin{equation} A = \begin{bmatrix} 4 & 2 \\ 2 & 1 \\ ...
0
votes
0answers
30 views

Sparse coding with local sparseness of dictionary

The title is probably pretty unclear, I hope I am able to explain it better here. I am currently working on a problem in the field of sparse coding, that is Principal Component Analysis, Non-negative ...
8
votes
3answers
491 views

Derivative of nuclear norm

I'm trying to take the derivative of nuclear norm with respect to its argument. nuclear norm is defined in the following way: $$\|x\|_*=\mathrm{tr}(\sqrt{x^Tx})$$ I'm trying to calculate: ...
0
votes
1answer
79 views

Explain the convexity by looking the hessian matrix of a function

The hessian matrix of a function is given by, $$ H = \begin{bmatrix} a & b & c \\[0.3em] b & b & 0\\[0.3em] c & 0 & c \end{bmatrix} $$ where, ...
1
vote
1answer
84 views

How to find a positive semi-definite linear combination?

Suppose we are given two explicit symmetric matrices $X$ and $Y$ and we'd like to find a non-zero real linear combination $aX+bY$ that is positive semi-definite (if possible). Is there a way to go ...
0
votes
0answers
14 views

NNLS for under determined system

I have system of equations to solve Ax=B under x>=0. I read that Non Negative least Square(NNlS) algorithm proposed by Lawson and Hanson could solve this system for over determined case( number of ...
0
votes
1answer
53 views

Minimize $L_2$-norm of $x1-b$ where $x \in R, b \in R^n$

Minimize $L_2$-norm of $x1-b$ where $x \in R, b \in R^n$ $||x1-b||_2 \rightarrow ||x1-b||^2_2=||x1-b||^T||x1-b||$ (squaring $L_2$-norm doesn't change outcome and yields quadratic) ...
0
votes
1answer
98 views

Solution to a Quadratic Minimization with Norm Constraint

How do I solve the optimization problem \begin{align} &\min_{\mathbf{x}\in\mathbb{C}^N}\mathbf{x}^H\mathbf{A}\mathbf{x}+2\Re\{\mathbf{b}^H\mathbf{x}\} \\ \mbox{subject to }\\ ...
3
votes
2answers
124 views

Singular Values of Matrix as Optimization Problem

Assume that $A$ is a positive semidefinite symmetric matrix. It is known that $$\max_{||y||\leq1} \quad y^TAy$$ Has an analytical solution which is the maximum eigenvalue of $A$. This isn't hard ...
0
votes
1answer
99 views

SDP formulation of noisy low rank matrix completion (2)

Thank you Michael for the answering my previous question, SDP formulation of noisy low rank matrix completion. It seems that I overlooked the problems in my initial question. I didn't recognize the ...
1
vote
1answer
37 views

An equation related to covariance matrix, square root of the matrix, and Euclidean norm.

How can I prove this equation: $${ ({ x }^{ T }\Sigma x) }^{ 1/2 }={ \left\| { \Sigma }^{ 1/2 }x \right\| }_{ 2 }$$ In which $\Sigma $ is a covariance matrix. I tried some numerical examples in ...
0
votes
0answers
41 views

Isomorphisms of Convex Cones

A convex cone $C$ is a subsets $C \subseteq V$ of a vector space which is closed under positive linear combinations, i.e. for $\lambda, \mu > 0$ and $u,v \in C$ it is $\lambda u + \mu v \in C$. An ...
3
votes
2answers
537 views

Adding Elements to Diagonal of Symmetric Matrix to Ensure Positive Definiteness.

I have a symmetric matrix $A$, which has zeroes all along the diagonal i.e. $A_{ii}=0$. I cannot change the off diagonal elements of this matrix, I can only change the diagonal elements. I need this ...
1
vote
2answers
66 views

Hessian matrix and epigraphs

I'm working on a homework assignment concerning convex optimization and I came across a problem involving the convexity of the function and the convexity of the domain of the function. Consider the ...
0
votes
0answers
45 views

Approximation of largest eigenvalue

What is an approximation for the largest eigenvalue of a matrix $A $? I mean, I am looking for some expressions that can be used as approximation for largest eigenvalue
1
vote
0answers
195 views

Minimum L1 norm may not obtain the sparsest solution?

Here is a paper called For Most Large Underdetermined Systems of Equations, the Minimal L1-norm Near-Solution Approximates the Sparsest Near-Solution. However, I did not quite get its definition of ...
1
vote
1answer
94 views

minimum trace norm on the set of matrices with fixed diagonal entries

What is the min nuclear norm (sum of singular values) on all $n \times n$ matrices$A$ whose diagonal is fixed. i.e. $diag(A) = v$ Is it true that the diagonal matrix is a minimizer?
0
votes
1answer
58 views

Strictly positive solution of linear equations

Suppose $A\in\mathbb{R}^{m\times n}$, $b\in\mathbb{R}^m$, and $b\in \mathcal{R}(A)$. Show that there exists an $x$ satisfying $x \succcurlyeq 0$, $Ax = b$ if and only if there exists no ...
1
vote
2answers
192 views

Why the unit circle in $\mathbf{R^2}$ has one dimension?

When I was reading 'Convex Optimization, Stephen Boyd', I was wondering of following steps Consider the unit circle in $\mathbf{R^2}$, $i.e.$, $\{x\in\mathbf{R^2}|x^2_1+x^2_2=1\}$. Its affine hull ...