0
votes
1answer
13 views

Proximal operator fixed point property for matrices

$\newcommand{\prox}{\operatorname{prox}}$ $\newcommand{\argmin}{\operatorname{argmin}}$ $\newcommand{\dom}{\operatorname{dom}}$ Recall again that the proximal operator for vectors $\prox_{f}: R^n ...
3
votes
1answer
57 views

Rank one plus diagonal matrix approximation

Given $A \in R^{n \times n}$, $A$ symmetric. I'm trying to solve the following minimization problem: $\underset{u \in R^n, d \in R^n} \min \, \frac{1}{2} \|X - A\|_F^2$ subject to $X = u u^T + ...
0
votes
1answer
61 views

Solving the cost function optimization problem using linear programming

My cost function is in the form $$ \Delta u^T P \Delta u + q^T \Delta u$$How shall I put it in the form of $c^T x$ to be able to solve it using linear programming?
0
votes
1answer
184 views

How to calc $\min ||J\Delta\tau + D||_*$

How to calculate $$ \min_{\tau} ||J_1 \tau_1 + \cdots + J_p \tau_p + D ||_* $$ where $\tau_1, \cdots, \tau_p \in \mathbb{R}$ $J_1, \cdots, J_p, D \in \mathbb{R}^{m \times n}$ $||\cdot||_*$ is sum ...
1
vote
1answer
112 views

How to show this algorithm on positive semidefinite matrices converges to a global maximum determinant

I'm dealing with an algorithm which is supposed to converge to the maximum determinant of certain positive semidefinite matrices. The problem is that we have such a matrix, and we vary certain ...
2
votes
1answer
325 views

How to find closest positive definite matrix of non-symmetric matrix

I have a matrix A given and I want to find the matrix B which is closest to A in the frobenius norm and is positiv definite. B does not need to be symmetric. I found a lot of solutions if the input ...
1
vote
0answers
166 views

Minimum L1 norm may not obtain the sparsest solution?

Here is a paper called For Most Large Underdetermined Systems of Equations, the Minimal L1-norm Near-Solution Approximates the Sparsest Near-Solution. However, I did not quite get its definition of ...
2
votes
1answer
49 views

How to prove a set is convex

Let $E = \{x\mid (x - c)^{T} P^{-1} (x-c) \le 1 \}$, where $P$ is symmetric positive definite. Show that $E$ is convex. Here is what I did. It seems like $E$ is an ellipsoid. We want to show that ...
0
votes
0answers
33 views

Optimization objective modification for matrix factorization.

I have the following optimization objective: Here A is an mXn user-item rating matrix. W is a nXn weight vector that I am trying to learn. beta and lambda are parameters passed in. ...
2
votes
0answers
31 views

Semidefinite Program formulation

I have the following problem and would like to formulate that as an SDP. I am not sure how to approach this : A set $S$ is given such that : $$ S = \{P \in R^{n \times m} : ||p_i - c_i|| \leq d_i ...
0
votes
2answers
51 views

Non-elementwise Matrix Derivatives

Let A,B,C,D,X be matrices. I'd like to perform a Gradient Descent minimization to the loss functin $$ tr[(AXBX^TC-D)^T(AXBX^TC-D)] $$ My question is, how to take the gradient efficiently w.r.t. $B$? ...
2
votes
0answers
63 views

Maximizing the smallest eigenvalue of a linear combination of matrices

I have an engineering back ground. Due to work, I came across this problem \begin{align} &\max_{\lambda,y_i\in \mathbb{R}}~\lambda \\\ ...
0
votes
1answer
59 views

Proximal operator

What is a proximal operator and how would one derive it in general for a function? In particular, if I had a function: $ f(x) = x^TQx + b^Tx + c $ How would I get the proximal operator for this if Q ...
5
votes
3answers
258 views

Is inverse matrix convex?

I wonder a generalization of Jensen's inequality: let $\mathbf{X,Y}$ be two positive definite matrices, can we obtain the following Jensen like inequality ...
1
vote
1answer
215 views

Convexity of the squared Frobenius norm of a matrix

I was reading this paper where the define an optimization problem as where K and L are kernel matrices and $\pi$ is the permutation matrix. They have explained that the function is convex because ...
1
vote
0answers
38 views

Looking for a “Neat” Transform to Yield a Convex Set

Optimizing on a unit sphere $\mathbb{S}^n$ is almost a convex problem (if the function is convex in the new set) if we make our "new" set $\mathbb{R}^n$, via the stereographic projection. Clearly ...
1
vote
1answer
64 views

Is this function involving matrices convex?

Let $X\in \mathbb{R}^{n \times n}$. Then, is the function $$ \text{Tr}\left( (X^T X )^{-1} \right)$$ convex in $X$? ($\text{Tr}$ denotes the trace operator)
3
votes
1answer
127 views

optimizing a logdet function with respect to a scalar and the Hessian matrix

Given a logdet function $\mathcal{L}(\gamma)$, $$ \mathcal{L}(\gamma) = \log\vert \mathbf{I} + \gamma\mathbf{S} \vert - \mathbf{q}^T(\gamma^{-1}\mathbf{I} + \mathbf{S})^{-1} \mathbf{q}, $$ where ...
2
votes
2answers
179 views

Optimize a log det function with respect to a matrix, and the saddle point analysis

Suppose I want to to find the local minima of a logdet function $\mathcal{L}$ with respect to a Matrix $\mathbf{A}$, $$ \mathcal{L} = \log\vert \mathbf{I} + \mathbf{A}\mathbf{S} \vert - ...
1
vote
1answer
146 views

Why does the non-negative matrix factorization problem non-convex?

Supposing $\mathbf{X}\in\mathbb{R}_+^{m\times n}$, $\mathbf{Y}\in\mathbb{R}_+^{m\times r}$, $\mathbf{W}\in\mathbb{R}_+^{r\times n}$, the non-negative matrix factorization problem is defined as: ...
0
votes
1answer
261 views

Positive semidefinite Matrix examples query

This might be really dumb question but I've just started dealing with such matrices. I would like to know why $$\begin{bmatrix} 0 & 1 \\ 1 & x \end{bmatrix}$$ cannot be a positive semidefinite ...
0
votes
1answer
71 views

Rewriting a quadratic Matrix equation as a quadratic vector equation

Consider the set of $N \times N$ matrices $\{W_i\}_{i=1}^{i=L}$, set of $N \times 1$vectors $\{g_i\}_{i=1}^{i=L}$ and $\{h_i\}_{i=1}^{i=L}$. Now consider the following sum \begin{align} ...
0
votes
1answer
240 views

relation between size of matrix and condition number

I have a matrix A of size NxM. Is there any relationship between size of a matrix A with the condition number ? I am computing the pseudo inverse (pinv in matlab ) ...
2
votes
1answer
74 views

Proof for certain matrix results?

There are certain results of matrices that Stephen Boyd uses often in his book on Convex optimization. Can someone provide me proof for the results I have enumerated below: If $B \in S^n$ and $A \in ...
0
votes
2answers
209 views
3
votes
1answer
102 views

decomposing PSD block matrix into two PSD block matrices

Given $Q = \left( \begin{array}{ccc} A + B & C \\ C^T & D\end{array} \right) $, where we know that $Q, A, B, D$ are all positive semi-definite, square, but not necessarily equally sized ...
2
votes
0answers
156 views

Optimization problem about large matrices

I'd like to solve the following optimization problem: Find non-negative scalar $a$, $b$, $c$ to minimize $\| (D-(aA+bB+cC+D^{-1})^{-1})y\|^2+2\operatorname{trace}((aA+bB+cC+D^{-1})^{-1})$ where ...
1
vote
2answers
383 views

How to prove a set of positive semi definite matrices forms a convex set?

Let $C$ be the set of positive semi-definite matrices, how can I prove it is a convex set?
2
votes
2answers
371 views

Linear optimization problem: Minimizing a linear function over an affine set.

The problem is as follows: Give an explicit solution of the linear optimization problem below. $$ \text{minimize}\ c^Tx \\ \text{subject to}\ Ax\ =\ b $$ No other information is given. My ...
3
votes
2answers
80 views

About the convexity of Ky Fan's norm

As we know, the Ky Fan norm is convex, and so is the Ky Fan k-norm. My question is, does this imply that the difference between them is a non-convex function, since it results from "difference between ...
2
votes
1answer
2k views

Armijo's rule line search

I have read a paper (http://www.seas.upenn.edu/~taskar/pubs/aistats09.pdf) which describes a way to solve an optimization problem involving Armijo's rule, cf. p363 eq 13. The variable is $\beta$ ...
0
votes
2answers
70 views

Any idea which matrix theorem this is?

I came across a theorem that boyd uses to convert the simplex to the form of a polyhedra. I don't know anything about this theorem. Theorem states: If $B$ has rank $k$, then we can find two matrices ...
2
votes
2answers
73 views

Geometric difference between $x^TAx$ and $x^TAx + b^Tx + c$

What is the difference between $x^TAx$ and $x^TAx + b^Tx + c$ geometrically? Some analogous examples from quadriatic equations would be great.
1
vote
1answer
75 views

On the convexity of the element-wise norm 1 of a pseudoinverse

Let us define $\|A\|_1$ the element wise norm 1 of a matrix $A \in \mathbb{R}^{n \times m}$ as $$ \|A\|_1= \sum_{i,j} |A_{i,j}|. $$ Obviously, this function is convex over $\mathbb{R}^{n \times m}$. ...
0
votes
1answer
54 views

matrix completion by rank minimization

In matrix completion, the starting point is often stated as: the optimization problem for matrix completion: min(X): (1/2) ||X-M||^2 s.t. rank(X)<= r Where X is the reconstructed matrix and M ...
0
votes
1answer
143 views

Matrix computations problem: rank, pseudo inverse,…

Suppose we are given two arbitrary $m \times n$ matrices, $A$, $B$, where we know $B$ has full column rank. Let $m>>n$. Can we always find a square $m \times m $ matrix $X$, such that $A=XB$? I ...
1
vote
1answer
115 views

Convex Combination of Hermitian Matrices

Assume all the matrices I discuss about are $N \times N$. Consider any two hermitian matrices $A_1$ and $A_2$ which are indefinite. The question is, In general, for any $A_1$ and $A_2$, does there ...
2
votes
1answer
142 views

Positive values for a set of quadratic forms of Hermitian Matrices. (To find a set of vectors in which a hermitian matrix is positive definite)

Assume all matrices I discuss about are $N \times N$ and the vectors conform with dimensions. Consider the following set of Quadratic inequalities where all the matrices $A_i$ are hermitian. ...
1
vote
2answers
389 views

Proof of Convexity?

Given a positive semidefinite matrix $A$, is $\operatorname{Tr}X^TAX$ a convex function in $X$? Am looking for a proof of convexity or non-convexity, whichever is true.
0
votes
1answer
144 views

Correctly adding constraints to Ax=b

I have a function of the form $$ E(\mathbf{x})=E_1(\mathbf{x})+E_2(\mathbf{x}) =\sum_i\|\ldots\|^2+\sum_j\|\ldots\|^2 $$ and want to solve the optimization problem $$ ...
2
votes
1answer
680 views

Maximum likelihood covariance estimation of Gaussian

I was reading these notes on matrix calculus http://research.microsoft.com/en-us/um/people/minka/papers/matrix/minka-matrix.pdf and I could not figure out how to go from equation (30) to (31). Any ...
2
votes
1answer
179 views

Non-negative solution to matrix equation

I want to solve $Ax = b$ subject to the constraint that all of the elements of $x$ are non-negative. If such a solution does not exist, I want to find non-negative $x$ such that the quadratic form ...