A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

1
vote
0answers
9 views

Basis pursuit denoising

The Lagrangian form of basis pursuit denoising $\min_{w} ||w||_{\ell_{1}} + \lambda ||Aw-x||_{\ell_{2}}^{2}$ can be solved using proximal gradient descent. Proximal methods also can be used to ...
2
votes
1answer
19 views

Showing that $x_{k+2}$ is a point which approximates a maximum?

Suppose that $x_k<x_{k+1}$ and $f'(x_k)>f'(x_{k+1})$. How can I show that the secant method will give $x_{k+2}$ as a point which approximates a maximum? $$x_{k+2}= ...
0
votes
0answers
36 views

how can I find the minimum of $f(x)$ by bisection method?

what will happen if the bisection method is applied in the range $a\le x\le b$ when $f(x)$ does not have a minimum in this range?
-1
votes
0answers
11 views

In Fermat-Weber's problem, show that if $x$ is not an anchor point, then $\nabla f(x)$ will exist

I have this problem which I think I have kind of solved, but I don't know how to calculate the derivative of the following $f(x)$ without using the formula $\nabla ||u||=\frac{u}{||u||}$ and I also ...
0
votes
0answers
5 views

is multiobjective optimization easier with fewer objectives?

it seems to me that optimization on nonconvex problems can usually reach better results with fewer criteria in the objective function. for example, in a given block of search time, if i am asking for ...
1
vote
0answers
23 views

Minimum of the difference of two logarithms

I am trying to find an analytical expression of the minimum of $$ f_n(x) = \frac{2x}{n^2+n}\log(x) - \frac{2x+2}{n^2+3n+2}\log(x+1) $$ when $x\in [1;n]$ I used to think from graphing it that this ...
0
votes
0answers
26 views

Strict Feasibility in Interior Point Methods

As we know, in the interior point methods, all the iterates have to be strictly feasible. I implemented an affine scaling interior point for nonlinear objective functions. For small examples (2D), it ...
0
votes
0answers
21 views

Indicator Variable, Mixed Integer Linear Programming

Assume $x$ is a real variable, and $0\leq x \leq1$. Besides, $y$ is a binary random variable. I need a linear program that: if $y$ is $1$: $x>0$, if $y$ is $0$: $x=0$ I know the following ...
0
votes
1answer
27 views

is this a convex optimization problem?

Can someone clarify is this a convex optimization problem or not. $min \| X-UV\|_{F}\quad $ s.t $ \quad U \geq ,V\geq0$ . If not , what makes the problem non-convex?
0
votes
1answer
49 views

How to perform the optimization when gradient is a matrix $\mathbf{R}^{n\times n}$

I am trying to optimize this cost function by using Gauss-Newton method. $$f = \sum_{i = 1}^n Tr{(Z^TZ)}$$ where $Z$ is a $4\times4$ matrix and it is a function of real vector ...
0
votes
0answers
14 views

relationship of eigenvalues of a matrix with conjugate gradient method

Assume that $Q$ has all its eigenvalues in the two intervals $[a,b]$, $[a+\delta,b+\delta]$, while $a,b,\delta>0$. Show that for every start point $x_0$, after two steps of conjugate gradient ...
1
vote
0answers
24 views

Max operator in optimization

If this question is already discussed, I am sorry for putting it again. I have a non-linear optimization problem as follows (this is portfolio optimization problem with mean $J_M$ and semi variance ...
0
votes
0answers
41 views

Nonsmooth optimization

Now I have a chance taking a course in nonsmooth optimization, the course outline writes: convex analysis, subdifferential calculus and proximal mapping. various numerical algorithms to solve ...
1
vote
1answer
20 views

Proving a function of matrix is convex

I have a function of a matrix and a vector $f(A,b)=y^\top (I-A)^{-1} b$ and I want to know the conditions under which it is convex. For functions of a vector, the positive definiteness of the Hessian ...
1
vote
1answer
17 views

Solve system of non-linear differential equations

I have a large system of non-linear differential equations of the form: $x_{i}''(y) = f_{i}(x_{1}, x_{2},\ldots,x_{n},x_{1}', x_{2}',\ldots,x_{n}', x_{1}'', ...
0
votes
0answers
21 views

Primal-dual Newton Barrier Method for Nonlinear Functions

I am writing the code for primal-dual Newton barrier method for nonlinear functions (algorithm described here in section 2). I have a few questions about the implementation details in the paper. The ...
0
votes
0answers
17 views

Intuition for gradient descent with Nesterov momentum

A clear article on Nesterov’s Accelerated Gradient Descent (S. Bubeck, April 2013) says The intuition behind the algorithm is quite difficult to grasp, and unfortunately the analysis will not be ...
1
vote
1answer
24 views

Example of maximal monotone operators in non-reflexive Banach spaces with applications in PDE

My question is about examples of maximal monotone operators that are defined in non-reflexive Banach spaces and have applications in PDEs, variational inequalities, etc (any application actually)? If ...
0
votes
0answers
35 views

L_1 norm optimization as a sequence of linear optimizations?

Does someone know of numerical methods to approximately solve ${\bf x_0} = \min_{\bf x}\{ \left\|\bf Mx - b\right\|_1\}$ by using some sequence of linear optimizations? Links or ideas are both ...
1
vote
1answer
26 views

Difference: Newton's method, Newton-Rhapson method, Gauss Newton-method.

I would appreciate some clarification w.r.t. algorithms for solving nonlinear systems of equations. 1 - I don't understand the difference between Newton's method and Newton-Rhapson method. In [1], ...
1
vote
0answers
20 views

Penalty methods in Unconstrained problems

Let me define $p : \mathbb{R}^n \to \mathbb{R}$ as a penalty function for the feasible set $\mathcal F$ of an equality and inequality constrained problem $(P)$ $$(P) : \text{min}\space f(x) \space ...
0
votes
0answers
17 views

Nonlinear least squares problems with binary variables

I want to solve the heat equation $T_t(x,t) = - L_x . T(x,t) + F(x,t)$ in an edge-weighted graph where $L_x = \sum_i x_i e_{ij}$ is weighted Laplacian matrix of the graph. Then I conclude to the ...
1
vote
0answers
17 views

Convexity Proof with constraints on the gradient

Consider a minimization problem $(P)$ : minimize $f(x)$ subject to $\delta_C(x) \leq 0$ Now assume that $\emptyset \neq C \subset \mathbb{R}^n$ is convex and let $f: \mathbb{R}^n \to \mathbb{R}$ be ...
0
votes
0answers
15 views

About the alternating optimization

The problem is defined as follows: $$ min_{A,B,C} f(A,B,C) $$ and the problem couldn't solve by gradient descent or close-form solution. Thus, the usual way is to use the alternating optimization: ...
1
vote
1answer
31 views

Convexity proof - can I get some pointers?

Prove that $C \subset \mathbb{R}^n$ is convex iff $\forall m \in \mathbb{N}$ and every set of $m$ points $\{x_1,...,x_m\} \subset C$ we have that $\sum_{i=1}^m \lambda_i x_i \in C$ Where ...
0
votes
0answers
11 views

How to solve a multidimensional stochastic optimization program with minimizing operators?

How to solve the following optimization program? $$ \max_{u_i^h,u_i^l,i=1,2}\Pi =E\big[a\min\{u_1^h,u_2^h, K_1^h,K_2^h\}+b\min\{u_1^h,u_2^l,K_1^h,K_2^l\}\\ ...
1
vote
1answer
41 views

Newton optimization algorithm with non-positive definite Hessian

In the newton optimization algorithm to find the local minimum $x^*$ of a non-linear function $f(x)$ with iteration sequence of $x_0 \rightarrow x_1 \rightarrow x_2 ... \rightarrow x^*$ all $\nabla ^2 ...
1
vote
1answer
28 views

Optimization of Frobenius Norm and Nuclear Norm

How to solve the following optimization problem, \begin{equation} \boldsymbol{\hat{x}} = argmin_{\boldsymbol{x}} \frac{1}{2} \| \boldsymbol{x - y} \|_F^2 + \lambda \| \boldsymbol{x} \|_{*} ...
1
vote
0answers
37 views

Difficulty in understanding a solution: Constraint minimization of sum of Non-symmetric matrices

I am trying to understand why there is significance difference in the performance of two proposed solutions. Original question (Constraint minimization of sum of Non-symmetric matrices) ...
0
votes
0answers
10 views

How to minimize $ \int_{\Omega} \frac1{2 \theta} (u(x) - v(x))^2 + \lambda|\rho(v(x))| dx $

I'm new in optimizations and i am trying to understand how to obtain $ v $ that minimizes $ \int_{\Omega} \frac1{2 \theta} (u(x) - v(x))^2 + \lambda|\rho(v(x))| dx $ where $\rho(x)$ - continuous ...
0
votes
1answer
38 views

Find $\min x^TAy+b^Tx+c^Ty$ subject to $1^Tx=1^Ty=1,x\ge 0,y\ge 0$

The problem seems to be easy but I can't find a solution :( Problem: Given $A\in\mathbb{R}^{m\times n}, A\ge 0, b\in\mathbb{R}^{m}, c\in\mathbb{R}^{n}$. Minimize $f(x,y) = x^TAy+b^Tx+c^Ty$ subject to ...
1
vote
1answer
30 views

how to find the solution of this cost function?

I have the following cost function. $J = \sum_{i=1}^N a\, Trace(W^TX_iW) - b\, Trace(W^TY_iW)$ Where $X_i$ and $Y_i$ are symmetric matrices, $a$ and $b$ are scalars. How can I find W?
0
votes
2answers
87 views

Constraint minimization of sum of Non-symmetric matrices

I am trying to find closed form solution to following problem \begin{equation} \begin{array}{c} \text{min} \hspace{4mm} \big(\lambda_1\left( \mathbf{y}^T V^{(1)}\mathbf{x} \right)^2 + ...
1
vote
0answers
12 views

Local optimization with multiple starting values \approx global optimization?

I need to find the minimum/maximum of a nonlinear function but the constraints in the optimization problem make it tougher to solve (not a convex problem). I don't have a good global optimization ...
0
votes
0answers
4 views

Finite difference method hessian matrix

In the finite difference method, after every iteration, do you have to update the hessian matrix and the grad function?
2
votes
0answers
15 views

Determining active constraints in KKT

Suppose there is a constrained optimization problem having inequality constraints. We can solve it using Karush-Kuhn-Tucker conditions. My question is how do we determine which constraints are active ...
0
votes
0answers
18 views

maintaining monotonicity in an optimization problem

I have written a program for optimizing a set of generators. I have hourly price and cost data and need to figure out when a generator should run or just stay off. I describe the problem in more ...
1
vote
1answer
19 views

Problems with vector vector derivative in optimization

I have a loss function of the followoing form: $L(\mathbf{a}) = \|\mathbf{b} - \mathbf{a}\|_2^2$ Where, $\mathbf{a}$ and $\mathbf{b}$ are vectors of dimension $d\times 1$. I need to calculate ...
1
vote
0answers
23 views

Convexity over a line given a convex interval [duplicate]

Let $f : \mathbb{R}^n \to \mathbb{R}_∞$ be a function. I want to prove that $f$ is convex over the line $L_{v,x_0}$ iff $\psi : \mathbb{R} \to \mathbb{R}_∞$ $\psi(t) := f (x_0 + tv)$, is convex ...
1
vote
0answers
25 views

Name of method which includes Taylor linearization inside fixed point iteration

I read paper about Horn-Schunck multiscale method for computing optical flow Core part of this algorithm is minimizing some functional. One part of functional contains nonlinear term inside L2 norm. ...
0
votes
1answer
17 views

understanding a statement in Gill, Murray and Wright “Practical Optimization”

Hi: I'm reading the book "Practical Optimization" and there's a part in Chapter 3 that I can't prove to myself but I'm sure it's true. On page 64, they define the Taylor expansion of $F$ about ...
0
votes
0answers
16 views

Analytic minimization of linear algebraic expression with nonlinear constraint

I'm trying to solve the problem of minimizing $a x_1 + (1-a) x_2$ with constraint $b - b^2 \sqrt{(1-x_1)(1-x_2)} - \sqrt{x_1x_2}=0$ and where all variables lie in $(0,1)$. It comes out of a physics ...
1
vote
0answers
26 views

Efficient method for refining parameters in nonlinear curve fitting

I have time-series electrical current data $i(t)$ with transient steps in it which are convoluted with the hardware filter used in data acquisition. As a result, the real steps in current, which would ...
1
vote
1answer
26 views

fastest path between two points with current

This question is an extension of the following trivial problem: on a stationary body of water, a motorized device is 100m S and 10m E of a buoy. Given the device can move 5m/s in stationary water, ...
1
vote
0answers
15 views

Optimal design for constrained Bayesian slope intercept model

Here is a problem I've been stuck on for quite a while. Consider the model \begin{equation} \mathbf{y}=\mathbf{H}\pmb{ \theta }+\pmb{\epsilon }. \end{equation} The design matrix is given by: ...
1
vote
0answers
34 views

Is there any algorithm for finding the minimum distance to the complement of a convex set?

There have been some algorithms for finding the projection from a given point onto a convex set. This problem seems to be quit easy because of the convexity of the set. However, in the case of finding ...
0
votes
0answers
22 views

solver for non-convex matrix optimization with convex constraints

So here is the problem: $\max_{D} ~~ \|A+BD\|$ subject to $\|D\|<1$ (any norm you like) where matrices A and B are given. The cost function is evidently convex as well as the constraint, but ...
0
votes
0answers
44 views

Solution of a linearly constrained quadratic programming problem

What is the solution of the following optimization problem: \begin{align} &\min{\mathbf{p}^\mathrm{T} \mathbf{B} \mathbf{p}}\\ &\text{subject to}: \mathbf{0}\leq{\mathbf{p}}\leq \mathbf{1}. ...
0
votes
0answers
41 views

Maximizing a convex function outside a convex set?

I want to prove the following equality: \begin{equation*} \min_{x: x^2 \ge t - x} x^2 = \max_{0 \le \mu \le 1} \left( \mu t + \frac{\mu^2}{4 (\mu -1)} \right). \end{equation*} The objective function ...
1
vote
0answers
16 views

Nonlinear optimization with range constraint

I am trying to solve the following problem, but so far each time trying to program either one or more of the constraints are not satisfied (using Matlab). Phrasing the problem in two different ways, ...