A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

1
vote
0answers
12 views

Does a zero duality gap imply global optimality?

Let's say we are given a nonlinear optimization primal problem (P). Suppose that the dual problem (D) to the primal optimization problem (P) achieves a zero duality gap with a solution to the primal ...
0
votes
1answer
13 views

One definition of strong convexity (from textbook of Prof. Bertsekas in 2015)

In strong convexity, there are a few definitions, one of them is: $f$ is strongly convex over $\mathcal{C}$ with coefficient $\sigma$ if $\forall x,y \in \mathcal{C}$ and all $\alpha \in [0,1]$, we ...
1
vote
1answer
16 views

Is it possible to make a linear reformulation?

The question is what to do when we have a product of the three variables, quite different in their nature. One is binary, the second is real, and the third is from a discrete set of rational numbers. ...
0
votes
1answer
15 views

Question about least square allocation of an amount to different buckets.

Suppose we have to allocate $x$ amount to $k$ desired amounts. Is there algorithm to do this that minimizes the squared distance between the actual $k$ allocated values and the $k$ desired amounts? ...
0
votes
0answers
21 views

Optimization problem involving semidefinite matrix variable that is constrained to be a tensor product

I would like to solve the following optimization problem. With scalar $R$ and nine (mutually orthogonal) $9$-dimensional column vectors $\vec v_i$ all given ($\vec v_i\!'$ is the row vector Hermitian ...
0
votes
1answer
38 views

Find the minimum of a function for only positive values of the vector variable

Let variable vector $\vec{q}$ of size $m\times1$, and its diagonal counterpart $m\times m$ matrix $Q=diag(\vec{q})$, for some $m\in\mathbb{N}$. Define fixed parameter $n\times1$ vectors $\vec{p}, ...
0
votes
0answers
13 views

Relaxation of non-convex QCQP with one quadratic and one linear constraint

According to Boyd we know that a non-convex QCQP problem with one quadratic constraint has strong duality with the relaxed SDP or Lagrange counterpart. (check "Convex Optimization" by Boyd, Appendix ...
0
votes
1answer
24 views

A question about a proof in nonlinear programming book

I have a question about the proof of Proposition 1.2.1 (Stationarity of limit points for gradient methods) in the nonlinear programming book (2nd edition) by Bertsekas. At the beginning of the proof ...
3
votes
2answers
35 views

Distance between a plane and set of points

Suppose $m$ data points belonging to a class represented by matrix $A$. Therefore, the size of matrix $A$ is $m\times n$. In addition, suppose $w\cdot x + b=0$ be equation of a plane in ...
3
votes
0answers
66 views

If this problem is not unbounded, what's wrong in this dual derivation?

In a paper with 100 citation, Robust Support Vector Machine Training via Convex outlier Ablation, a convex relaxation is used. In this paper, a form of robust svm proposed: \begin{align} \min_{0\leq ...
1
vote
1answer
33 views

Function Optimization

Assume any recursive function like: (just for example, my rekursive function is just too big to write) $x_{n+1}=\frac{(x_{n}-3)^{5}x_{n}^{2}}{a\sqrt{x_{n}}}$ (or any other non-linear function) Is ...
0
votes
0answers
17 views

finite differences nonlinear least squares

I am facing to following nonlinear least-squares problem: $$\min_{u,\gamma} \frac{1}{1000} \int_{ \gamma(x,y)^2} + \int_{ [u(x,y)−u (x,y)]^2} + \int_{ [∆u(x,y)−\gamma(x,y)u(x,y)]^2}$$ where the ...
1
vote
0answers
14 views

How to solve the fractional polynomial optimization problem?

The optimization problem has a fractional of polynomials as the objective function, with linear constraint. For example, $\min\limits_{x,y}\quad ...
0
votes
1answer
24 views

Gradient of a sum of indicators

Say I have a function $\mathbb R^n \rightarrow \mathbb R$: $$f(w_1,\ldots,w_n) = n^-\sum_{i\in I^-}w_ix_i$$ with fixed $x_i\in\mathbb R$ (data), $I^-$ the set of indexes with negative sum operands ...
0
votes
0answers
24 views

Optimize to Find the Mahalanobis Distance to Minimize the Term

I have an optimization problem defined as following: Assuming we have a data set $ { \left\{ \left( {x}_{i}, {y}_{i} \right) \right\}}_{i = 1}^{N} $ where $ {x}_{i} \in {\mathbb{R}}^{d} $ and $ ...
-2
votes
0answers
25 views

This fractional quadratic optimization problem is non-Convex, why?

Why is the following function $f(x)$ non-convex? $$ f(x)=\min\frac{x^TQx}{x^TPx+1} $$ where $Q$ and $P$ are positive semi-definite matrices.
-1
votes
3answers
47 views

How to find a pareto optimal solution in a smart way (3 variables)

$\max\left( { 3x }_{ 1 }+4{ x }_{ 2 }+2{ x }_{ 3 } \right) $ ${ x }^2_{ 1 }+{ x }^2_{ 2 }+{ x }^2_{ 3 }\le 1 $ ${ x }_{ i }\ge 0 $ I have to find a Pareto Optimal solution, but I can't solve this ...
2
votes
0answers
20 views

Closest line to point after non-linear map

I have a map on a vector space $M(\vec{r})$, defined as below. All components (vectors, matrices, everything) are reals in the unit range $[0,1]$. The map $M(\vec r)$ is defined as the sum of an ...
0
votes
0answers
27 views

Can this problem be a form of Nonlinear Programming?

How we can reformulate blew problem as a form of nonlinear programming problems? $$ \begin{array}{ll} & \min&\frac{1}{x_1}+\frac{1}{x_2}+\cdots+\frac{1}{x_n} &\\ & ...
1
vote
0answers
54 views

Derivation of back-propagation equation $\frac{\partial E(\theta)}{\partial W^k}=x*\delta h^k+\tilde{h}^k*\delta y$ for convolutional autoencoders

I was reading the following paper on convolution stacked auto-encoders and they had the following convolution neural network (for auto-encoders, notice I didn't write the offset term [to avoid ...
0
votes
0answers
32 views

Bilinear Constraint

I would like to formulate the following Optimization problem. My question is focused on the constraint. Given a "typical" objective function, e.g.: $$ \min c^T v $$ s.t. $$ 0 = a_1 v_1 - a_2 v_2 + ...
1
vote
2answers
23 views

Bivariate optimal density

Consider any feasible $p:[0,1]^2\to [0,1]$ that allows discontinuities and the problem $$\min_{p(.)} \int_0^1\int_0^1 p(x,y)^2 dF(x) dG(y)$$ s.t. $$\int_0^1 p(x,y)dG(y)=k\phantom{0} for \phantom{0} ...
3
votes
1answer
54 views

Finding the root for a multivariate function.

Background In a practical problem I need to find the solution to: $$f(\bar{x}) - \bar{p} = \bar{0}$$ where $ f : \mathbb{R}^2 \rightarrow \mathbb{R}^2 $. I don't know the exact expression for $f$ ...
0
votes
0answers
18 views

Mapping sphere surface to a vector space such that distances are preserved?

I have a unit radius sphere (say in 3D) centered in origin. Thus the shortest distance between two points on the sphere is the geo-desic. Is there a transformation (linear or non-linear) on the points ...
0
votes
0answers
39 views

Shannon Entropy Maximization with Constraints

I have got a cumulative distribution function $F_X(x)=Pr(X<=x)$. This distribution is described by 2 parameters $\alpha, \beta$. We define $F_k$ as follows: $\forall k<=n_k, ...
1
vote
2answers
28 views

Would like some help formulating an optimization problem

I have a function $f$ that takes $n \geq 1$ positive real-valued arguments $\mathbf{a} \in R^n_+$. This function is defined for all amounts of inputs (e.g. $f(1)$ and $f(3, \pi, 17)$ are both valid) ...
0
votes
1answer
26 views

invert S shape logistic curve fitting

I have a function F(x) = a + b / (1 + exp(-(cx + d))) and a small data set containing 5 sample: x = [10.0, 5.0, 2.0, 0.5, 0.25] and y = [23, 24, 25, 26, 27]. How can I find the suitable parameters, ...
0
votes
0answers
10 views

Duality gap analysis

I solved a non-linear non-convex optimization problem via dual decomposition optimization using sub-gradient method. (my main goal is to solve the problem in a distributed way). I solve the same ...
1
vote
0answers
16 views

Convergence results for block coordinate descent methods

I am trying to solve the problem minimize $f(x)$ subject to $x_1 \in C_1, x_2\in C_2, ... x_m\in C_m$ where $x_1, ..., x_m$ are block subvectors of $x$, and $C_i$ are each closed convex sets (not ...
0
votes
1answer
13 views

Maximization: KKT on unbounded region

Solve the following NLP: $$\left\{\begin{matrix} \min & -3x+y-z^2\\ s.t& g(x,y,z)=x+y+z \leq 0\\ & h(x,y,z)=-x+2y+z^2z=0 \end{matrix}\right.$$ My attempt Using kkt conditions, we ...
0
votes
0answers
25 views

non-linearity and non-convexity

I am taking a course on linear regression online and it talks about the sum of square difference cost function and one of the points it makes is that the cost function is always convex i.e. it has ...
1
vote
1answer
37 views

Matrix norm in the objective of an optimization problem

I am stuck with the following optimization problem from research. The optimization problem have the following objective function: $\|Q-H\|_\infty$. Here $Q$ is a PSD matrix and $H$ is a symmetric ...
0
votes
1answer
24 views

Eliminate cases before calculting all KKT conditions

I have the following non linear programming to solve: $$\left\{\begin{matrix} \min & (x-3)^2 + (y-2)^2 \\ s.t. & x^2 +y^2 \leq 5 \\ & x+y\leq 3 \\ & x \geq 0\\ & y\geq 0 ...
0
votes
0answers
14 views

Non-linear least squares and Bundle Adjustment

In METHODS FOR NON-LINEAR LEAST SQUARES PROBLEMS, 2nd Edition, April 2004 by K. Madsen, H.B. Nielsen, O. Tingleff on page 17 it states: Given a $f: R^n \mapsto R^m$ with $m \geq n$ We want ...
2
votes
1answer
36 views

If $H$ is positive definite and $s^Ty>0$, then $s^THs-\frac{s^Tyy^Ts}{s^Ty+y^TH^{-1}y}\ne -1$

Let $H\in\mathbb{R}^{n\times n}$ be symmetric and positive definite $s,y\in\mathbb{R}^n$ with $s^Ty>0$ How can we show, that $$s^THs-\frac{s^Tyy^Ts}{s^Ty+y^TH^{-1}y}\ne -1\;?\tag{1}$$ ...
0
votes
0answers
13 views

For which values of $c_1, c_2$ and $c_3$ is (1, 2, -2) a local minimum

Consider the problem $$\left\{\begin{matrix} \min & x^2 -2xy + 2xz +y^2 + 4yz + z^2 + c_1x + c_2y + c_3z \\ s.t & g(x,y,z)=-x^2 -4xy - 4xz -2y^2 -4yz - 2z^2 + x -y+z+4 =0 \\ \; & ...
0
votes
1answer
14 views

Constrained Optimization: $\min x_1$

Consider the problem $$\left\{\begin{matrix}\min & x_1 \\ s.t & x_2 \geq 0 \\ \; & x_2 \leq x_1^3 \end{matrix}\right.$$ It is asked to find the minimum and show why this does not satisfy ...
2
votes
4answers
86 views

Minimization on compact region

I need to solve the minimization problem $$\begin{matrix} \min & x^2 + 2y^2 + 3z^2 \\ subject\;to & x^2 + y^2 + z^2 =1\\ \; & x+y+z=0 \end{matrix}$$ I was trying to verify the first ...
0
votes
0answers
16 views

Convex signal reconstruction for convex generator function?

Let $f : \mathbb{R} \mapsto \mathbb{R}$ be a convex (not affine) function and suppose that $y = f(x)$. We want to reconstruct the input $x$ for a given $y$, and the standard approach would be to ...
1
vote
0answers
36 views

What curved ramp transports a ball from (1,1) to (0,0) most quickly, under the acceleration of gravity, with no friction or air resistance?

An infinitisemally small ball is placed at the top of a ramp which has a height of 1m and ends 1m away horizontally. What is the optimal curve of the ramp to minimize time taken for the ball to reach ...
0
votes
0answers
29 views

Convex envelopes of bivariate functions

In order to convexify my nonlinear non-convex program I need convex envelopes for the function $(x/y)^2$, both x,y are positive. I am only aware of the convex envelopes of the type $xy$ from here ...
0
votes
0answers
14 views

Is there any non-linear optimization technique whose running time depends on the diameter of the underlying polytope(induced by constraints)

It is well known that the running time of the simplex algorithm depends on the diameter of the polytope induced by the constraints. Is there any non-linear optimization technique that also has this ...
0
votes
1answer
21 views

Optimization: maximizing nonconvex sum of product of constraints

I'm wondering if there is any way to convexify, approximate, and/or simplify the following problem. $\max. \sum_{k \in K} \prod_{i \in I} (a_{ik} x_{ik} + b_{ik})$ s.t. $x_{ik} \in [0,1]$ where ...
0
votes
0answers
9 views

How to generate feasible $H$-conjugate descent search directions in convex subset

If we want to minimize a quadratic function $f(x)=c^Tx+\frac12x^THx$ (where $H$ is a symmetric positive-semidefinite matrix) in a convex subset $C\subset\mathbb{R}^n$, then is it possible to generate ...
0
votes
1answer
44 views

How to linearize this constraint a summation of a product of a integer with a binary

I have to linearize the following constraint, $$ \sum_{i \in V_C} \sum_{j \in V} \sum_{k \in K} y_{ik} \cdot x_{ijk\ell} \leq I_\ell \qquad \forall \ell \in V_D $$ where $y$ is a integer variable ...
1
vote
1answer
18 views

Stability of Model Predictive Controller

I appreciate if you suggest a reference that discusses the stability of Model Predictive Controller for continuous-time nonlinear systems.
1
vote
0answers
48 views

Polynomial roots finding algorithm

My initial problem is a parameter estimation problem that is solved by minimining a least-square criterion with the Gauss-Newton algorithm. However finding a good initial iterate is very tedious. ...
2
votes
0answers
46 views

Solve constrained system of linear equations from samples of a reference function

I have a system of $2n$ linear equations in $2n$ unknowns represented by the standard matrix equation: $$Ax = b$$ Where the solution vector $x = (p_1, ..., p_n, q_1, ..., q_n)$ represents real ...
0
votes
0answers
25 views

Making projected search directions conjugate

I'm trying to implement a minimization process for the optimization problem: ...
0
votes
1answer
39 views

KKT conditions for a convex optimization problem with a L1-penalty and box constraints

I am having some trouble deriving / understanding optimality conditions for a convex optimization problem of the form: $$\begin{align} \min_{x\in\mathbb{R}^d}~ & f(x) + C.\|x\|_1 &\\ &x_i ...