A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

0
votes
0answers
46 views

optimization of nasty expression with nice symmetry between expressions

Consider the function $\ f(x,y,z,\rho_a,\rho_b)=$ $ \log \left(1+ (x+ \rho_ay)^2 + \frac{(z+ \rho_by)^2}{1+(x+ \rho_by)^2} \right)+ \log\left(1+ (x+ \rho_by)^2 + \frac{(z+ \rho_ay)^2}{1+(x+ ...
4
votes
1answer
151 views

Lipschitz continuity of parametric optimizer

Consider the parametric optimal solution $x^{*}: \mathbb{R}^n \rightarrow \mathbb{R}^n$ defined as $$ x^*( y ) := \arg\min_{x \in X } \ \ x^\top x + x^\top A y \\ \quad \qquad \text{subject to: } \ ...
1
vote
0answers
38 views

When result of max of min problem is equal to min of max problem

Let's assume there are two functions $f(x)$ and $g(x)$. I want to know when the optimal $x$ of max of min of $f(x)$ and $g(x)$ is not equal to optimal $x$ of min of max of $\frac{1}{f(x)}$ and ...
1
vote
2answers
49 views

Is the set of all projection matrices a convex set?

The set $\phi=\{P| P^2=P\}$ contains all projection matrix. Is this set $\phi$ convex?
0
votes
1answer
32 views

Solving coupled non-linear equations

I am struggling to understand what the following question requires me to do: I believe I need to differentiate implicitly, but am unsure how I show it cannot be done.
1
vote
0answers
35 views

Explain KKT conditions without reference to duality.

Is it possible to explain (not derive) KKT necessary conditions without reference to the concept of Lagrangian duality?
0
votes
0answers
34 views

Constrained non-linear optimization problem

For some background, this comes from a sample size allocation problem in statistics. I am trying to minimize the following function (a sum of three variances), and could use some help with direction ...
1
vote
0answers
57 views

Saddle point problem (KKT) with block-diagonal matrix

Consider the following saddle point problem originating from an interior-point method algorithm: $$ \begin{bmatrix}\mathbf{H} & \mathbf{A}^{T}\\ \mathbf{A} & \mathbf{0} ...
19
votes
2answers
501 views

Maximizing curious symmetric function from simple combinatorics

A curious symmetric function crossed my way in some quantum mechanics calculations, and I'm interested its maximum value (for which I do have a conjecture). (This question has been posted at ...
0
votes
0answers
26 views

Non linear Programming Problem

I am struggling with the following question: Solve the following programing problem: max $f(x_1,x_2)$= $ \sqrt{(x_1 + 1) (x_2+1)} $ subject to $x_2-(x_1-1)^2 \leq 0 $; $x_1+x_2 \leq 7 $; $x_1, x_2 ...
0
votes
0answers
27 views

Separate a list of spheres into several lists, each contained in a sphere with a radius no larger than specified.

I have a list of arbitrary spheres, what I want to end up with is that list separated into a number of groups, where spheres in each group all fit into thier specific larger sphere. The limitation is, ...
0
votes
0answers
101 views

Fenchel dual vs Lagrange dual

Consider the Fenchel dual and the Lagrangian dual. Are these duals equivalent? In other words, is using one of the these duals (say for solving an optimization), would give the same answer as using ...
1
vote
1answer
63 views

Question regarding KKT conditions in optimization

Following is Proposition 3.3.7 in Bersekas' Nonlinear Programming. Let $x^*$ be the local minimum of the problem: $$\text{Minimize }\; f(x) $$ $$ \text{subject to: }\ h_j(x) = 0, ...
7
votes
0answers
98 views

Consider the problem minimize $f(x)= x^4 −1. $

I am studying for a test and I found this problem in the textbook that I'm using, there may be a conceptual problem that I'm coming across. My test is on unconstrained optimization (multi-variable) ...
1
vote
0answers
47 views

Solving many independent non-linear systems simultaneously

I'm working on solving lots of systems of nonlinear equations. Luckily, the non-linear equation is the same, but the parameters are different: $$ f(\vec{x}_0; c_0) = 0\\ f(\vec{x}_1; c_1) = 0\\ ...
2
votes
1answer
111 views

min : sum of L2 norm and squared-L2 norm.

Is there a closed-form solution of the following convex problem: $$\min_x \| x - u \| + C \| x - v \|^2$$ where $\| \cdot \|$ is the L2 norm.
1
vote
2answers
30 views

How to mathematically prove the optimality conditions for a univariate function?

Consider a univariate function $f(x)$. I know the graphical intuition behind why $f'(x)=0$ at the extrema of $f$. But how do you prove it mathematically? I start with the assumption of $x^*$ being a ...
1
vote
1answer
54 views

Why don't we go beyond the Hessian in multivariate optimization?

In univariate optimization, we perform the first derivative test to identify stationary points and the second derivative test to classify the stationary points as minima, maxima and inconclusive. When ...
1
vote
1answer
85 views

Constrained Optimization : Minimize sum of dot products

I am working on a problem to minimize sum of dot product. The problem can be stated as following. Given a matrix where each element is either 0 or 1. $$ \ A_{ij} = \{0,1\}; $$ with the constraint ...
2
votes
0answers
66 views

Constrained non-linear Optimization using Newton's method - Portfolio optimization

I want to solve following constrained optimization problem from portfolio optimization: The solution is supposed to be a modified risk parity portfolio: The optimization problem is: \begin{align} ...
1
vote
0answers
93 views

How can I find the unit vector that minimizes the number of nonzero projections that a set of points has on it?

$\underset{\mathbf{w}}{\min} ~ \|\mathbf{X}^T\mathbf{w}\|_1~~~\text{subject to:}~ \|\mathbf{w}\|_2^2=1$ where $\mathbf{X}\in\mathbb{R}^{d\times m}$ is a set of $d$-dimensional points and $m>d$. ...
0
votes
1answer
48 views

Minimizing a non-convex rational function of two variables

I need to minimize the following function $$f(x,y)= \frac{a}{x}+\frac{bx}{y}+\frac{cy}{x}+dy+\frac{e}{y}$$ where $a,b,c,d$, and $e$ are positive constants, and $x$ and $y$ are both strictly positive. ...
2
votes
0answers
64 views

Constrained Quadratic Optimization(Reproducing Kernel)

I am attempting to use a constrained quadratic optimization to find the coefficients of a reproducing kernel. The problem is as follows: $y(t)=\sum_{i=0}^J\alpha_iK(t, t_i)$ $Q(\alpha)= ...
1
vote
2answers
27 views

Solving for gradient of Frobenius norm term

Let's first define a couple of variables: $A,B,C \in \mathbb{R}^{m \times n}, D \in \mathbb{R}^{n \times n}$, and $\mu$ is a scalar. Say I have an ADMM sub-problem that looks like this: $\arg ...
1
vote
1answer
151 views

How to solve this optimization problem with equality constraints?

I want to find $\delta_j$ in the following optimization problem. My variables are $\gamma_i$ and $\delta_j$ (all other symbols are known parameters). Assume $i\in\{1,\ldots,9\}$ and ...
0
votes
0answers
20 views

Non-linear optimization with unknown derivative and real numbers only

Background I'm trying to optimize a set of 7 parameters which are the core configuration of some external engine (specifically Solr parameters). I already have an optimization function which grades ...
1
vote
0answers
28 views

Non-convex constraint made cost

Consider the non-convex optimization problem $$ \min_{x \in X} \ f(x) \quad \text{s.t.:} \ \ g(x) \leq 0, \ h(x) = 0 $$ where $X \subset \mathbb{R}^{2n}$ is compact and convex, $f$ and $g$ are ...
2
votes
1answer
43 views

Maximizing the uniformity of density function subject to moment constraints

Background I want to find a probability measure for a continuous random variable, subject to moment constraints, that is maximally "uniform", as defined below: Definition: Maximally Uniform ...
0
votes
0answers
40 views

Concave Quadratic Program

Let $X \subset \mathbb{R}^n$ be compact and convex. Consider $$ x^* := \arg\min_{x \in X} x^\top Q x + c^\top x $$ where $Q \prec 0$. I am wondering if there are cases where $x^*$ can be written as ...
1
vote
0answers
33 views

Hint for KKT Optimization problem

Can anyone help me with the following optimization problem please? I have to find the $\max f(c,y_1^1,\cdots,y_{N-1}^1,\cdots,y_1^M,\cdots,y_{N-1}^M)=c$ subject to the constraints ...
0
votes
1answer
80 views

Convex optimization approximation

Consider the optimization problem $\mathcal{P}_0$ $$ \min_{x \in \mathbb{R}^2} \left\| x-p \right\|^2 $$ $$ \text{sub. to: } \ A x \leq b, \ \ x_1^2 + x_2^2 = 1 $$ where $p \in \mathbb{R}^2$ is a ...
0
votes
1answer
64 views

Model $\min \frac{1}{2} \parallel Ax-B \parallel_2 + \lambda_1 \parallel Cx \parallel_1 + \lambda_2 \parallel Dx \parallel_\infty $ into standard form

I need to solve the following convex optimization problem: $\min \frac{1}{2} \parallel Ax-B \parallel_2 + \lambda_1 \parallel Cx \parallel_1 + \lambda_2 \parallel Dx \parallel_\infty$ s.t $x ...
1
vote
1answer
47 views

Minimize $\ell_1$ norm subject to $\ell_2$ constraint

I am trying to solve the following optimization problem: $$\min_{\|Px\|_2=1} \|x\|_1$$ I know it is non-convex. But some non-convex problems are still solvable. Update $P$ is 2x3. $x$ is a ...
3
votes
0answers
150 views

Finding optimal velocity profile using Dynamic Programming

This question has been asked on scicomp but I thought maybe it's more a mathematical problem of how Bellman's idea is to be applied here. The main problem for me is: How to introduce the time ...
2
votes
0answers
164 views

Levenberg's original article “A method for the solution of certain problems in least squares”

Does there exist any digital copy of the original article (or a transcript) K. Levenberg, A method for the solution of certain problems in least-squares, Quart. Appl. Math. 2 (1944): 164-168? It is ...
0
votes
0answers
43 views

Constrained optimization using a cutting plane on a tetrahedron

Consider the figure below where $(a,b,c,d)$ is a tetrahedron and $p=(1-t)a+tb$ is a point on the $ab$ segment. If $n_a$ and $n_b$ are two unit vectors associated with $a$ and $b$, respectively, then ...
0
votes
1answer
50 views

Evolutionary algorithm

Can someone provide me a good reference for the CMA-ES algorithm? I'm new in the world of optimization and just reading the author reference doesn't help me a lot. I know the basic idea of a genetic ...
3
votes
2answers
243 views

Maximizing “log det + log sum exp” function

I'm trying to find a numerical solution to the following optimization problem $$ \text{maximize } f(M) = \frac{1}{2} \log \det(M) + \log \sum_{i=1}^n \exp \left\{ - \frac{1}{2} x_i^T M x_i + a_i ...
0
votes
0answers
21 views

A non-linear optimization problem involving logarithms

I have a log likelihood function that I'm trying to minimize: $-\log(L) = -(\sum_i^n{c_i \log(a_i x + b_i y)} - \sum_i^n{\log(c_i!)} - x \sum_i^n{a_i} - y \sum_i^n{b_i})$ The function is only ...
1
vote
1answer
48 views

Optimization problems on the circle

Consider the optimization problem $$ \min_{x \in \mathbb{R}^2} x^{\top} P x + q^{\top} x$$ subject to: $$ A x = b, \ x \in X, \ x_1^2 + x_2^2 = 1$$ where $X$ is compact and convex. Then ...
1
vote
1answer
42 views

Analysing/Visualising shape of multi-variate function.

I have an unknown function $f:\mathbb{R}^2\rightarrow \mathbb{R}^2$ for which I'm determining a first order Taylor approximation through a non-linear optimization process in six variables (the ...
0
votes
1answer
56 views

project a point onto the intersection of surfaces

I have several non linear equations $g_i$ that represent surfaces $s_i$. Their intersection form the surface $S$. For example $s_1 : g_1(x_1,x_2,...,x_n)=c_1$ ... $s_n : g_m(x_1,x_2,...,x_n)=c_m$ ...
3
votes
2answers
52 views

Max of $3$-Variable Function

I'm trying the find the maximum of the function $$f(a,b,c)=\frac{a+b+c-\sqrt{a^2+b^2+c^2}}{\sqrt{ab}+\sqrt{bc}+\sqrt{ca}}$$ for all nonnegative real numbers $a, b, c$ with $ab + bc + ca > 0$. I ...
0
votes
1answer
45 views

Armijo conditions vs Reduction Conditions in Non-Linear Line Search

Overview Line search typically consists of four stages: Direction: Search direction Initial Step Size: length to search along the line on the first sub-iteration Bracket: find an interval along the ...
0
votes
1answer
121 views

Finding Shortest distance between a Sphere and Ellipsoid?

Suppose that ,I have a Sphere and an ellipsoid as Sphere: $(x-x_1)^2 + (y-y_1)^2 + (z-z_1)^2 = R_1^2$ Ellipsoid: $\large\frac{(x-x_2)^2}{a^2} + \frac{(y-y_2)^2}{b^2} + \frac{(z-z_2)^2}{c^2} = 1$ ...
3
votes
0answers
38 views

Solve non-linear equation with Matrices

I'm looking more for hints than specific answers, although I would be extremely grateful if provided with one. The problem I have is as follows: $$ -\Sigma (A+\Lambda_1)+I=0 $$ Here A is a constant, ...
0
votes
1answer
30 views

squaring the equality constraints

When creating an unconstrained optimization problem from an equality constrained one, the usual way to build the Lagrangian, is by adding a term consisting of a multiplier, multiplied by the equality ...
1
vote
1answer
36 views

Quadratic Problen with 2 constraints

Could someone help me to solve the following: $\min x^Tx$ s.t. $x^T a=1$ $x^T b=0$ where $x$,$a$ and $b$ are $(N\times1)$ vectors and $1$ and $0$ scalars. Thank you!
1
vote
0answers
44 views

Gradient descent via polynomial approximation

It seems that most proofs of convergence for gradient descent algorithms rely on strong conditions on the first and second derivatives of the function, for instance that $$|f''(x)| \leq K$$ over the ...
1
vote
1answer
94 views

Non-linear least squares with two dependent variables

I have data in the form $(t_i,x_i,y_i)$, i.e. position in 2D as a function of time. I have non-linear equations which I want to fit to the data. They give me a position $(X,Y)$ as a function of time ...