A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

4
votes
1answer
2k views

Linearization of a product of two decision variables

I am trying to solve a problem that involves constraints in which products of two decision variables appear. So far, I read that such products can be reformulated to a difference of two quadratic ...
4
votes
3answers
108 views

How do one solve a nonlinear combinatoric problem?

I am an undergraduate CS student and I am struggling with a problem. $Qx = b$ where $Q$ is a constant $m \times n$ matrix (with $m>n$), $x$ is a $n \times 1$ vector and $b$ is a $m\times 1$ vector....
4
votes
2answers
814 views

Analog of Simplex Method for Quadratic Programming

It happened so I need to work with an algorithm for solving QP problems. The main issue here is that I can't find any references to this algorithms (at least no references in sources in english). ...
4
votes
1answer
54 views

How can I experiment with Lagrange multiplier in QCQP?

Suppose we want to solve following optimization problem (it is a PCA problem in this post) $$ \underset{\mathbf w}{\text{maximize}}~~ \mathbf w^\top \mathbf{Cw} \\ \text{s.t.}~~~~~~ \mathbf w^\top \...
4
votes
1answer
90 views

Show that $(a + b)^4 + (a + c)^4 + (a + d)^4 + (b + c)^4 + (b + d)^4 + (c + d)^4 \le 6$ for $a^2 + b^2 + c^2 + d^2 = 1$.

For $a, b, c, d \in \Bbb R$ such that $a^2 + b^2 + c^2 + d^2 = 1$, show that $$(a + b)^4 + (a + c)^4 + (a + d)^4 + (b + c)^4 + (b + d)^4 + (c + d)^4 \le 6.$$ The answer uses the mysterious identity $$...
4
votes
0answers
71 views

Nature of the Hessian of the dual function?

I originally posted this over at MathOverflow but it did not receive much (...any) attention. I'm hoping someone can point me in the right direction over here. Consider a nonlinear optimization ...
4
votes
1answer
147 views

Gradient of the dual function for a nonlinear program

I'm attempting to find a proof for a property from Floudas' Nonlinear and Mixed-Integer Optimization book. Consider a nonlinear optimization problem of the form \begin{align} \min_{{\bf x}}&\quad ...
4
votes
0answers
85 views

How to use the Karush–Kuhn–Tucker conditions?

From what I read, the Karush-Kuhn-Tucker conditions are a generalization of the Lagrange Multiplier Method. For the Lagrange Multiplier Method I have been able to find a serie of steps I must do to ...
4
votes
1answer
62 views

An optimization problem involving a probability density function

I have three time-series $\mathbf{x}_{1}, \mathbf{x}_{2}, \mathbf{x}_{3}$. I would like to find a linear combination of the time series, that is, some scalars $a_{1},a_{2},a_{3}$ such that the sum $$\...
4
votes
0answers
49 views

Solving non-linear equations in a chosen subspace

I'm trying to find the root $\mathbf{f(x)=0}$ to the following sets of equations $$ f_1(x,y,z) = x^\prime - \frac{x}{\sqrt{x^2+y^2+z^2}} = 0 \\ f_2(x,y,z) = y^\prime - \frac{y}{\sqrt{x^2+y^2+z^2}} = ...
4
votes
1answer
75 views

QR-Decomposition of matrix valued function

I already posted the following question on MO, but id did not raise much interest there. Maybe the title is too elementary to gain research interest. Suppose I have a matrix valued function $$ F:...
4
votes
0answers
36 views

Nonlinear optimisation of Expectation

I am preparing for my exams and I can't get my head around the following question. I know there exists a general method for solving these problems but I don't know where to start. I would greatly ...
4
votes
0answers
320 views

Optimizing non linear programs of two variables

The scenario is; We've got $n$ stationary 360$^{\circ}$ sensors in an confined area (each sensor is located at some arbitrary $\left(x,y\right) = \left(x_{n},y_{n}\right)$), once a unit $t$ enters ...
4
votes
0answers
202 views

Why Compactness is Necessary at Minimax Theorem

According to Von Neumann's minimax theorem, I have $$\max_{x\in X} \min_{y\in Y}f(x,y)=\min_{y\in Y} \max_{x\in X} f(x,y)$$ for some compact sets $X$ and $Y$ and a convex (in $y$), concave (in $x$) ...
3
votes
3answers
95 views

A convex optimisation problem involving the Euclidean norm

Any ideas on how to approach the following optimisation problem? $$\begin{array}{ll} \text{maximize} & \|Ax\|_2^2+\|Bx\|_2^2+\|Cx\|_2^2 \\ \text{subject to} & \|x\|_2 = 1\end{array}$$
3
votes
2answers
76 views

$g$ is coercive for $g(x)=x^TAx+b^Tx+c$

Suppose $A$ is a symmetric positive definite matrix $A\in \Bbb{R}^{n\times n}$, $b \in \Bbb{R}^n$, and c is a real number. Let $$g(x)=x^TAx + b^Tx + c$$ Show that $g$ is coercive. Because $A$ is ...
3
votes
1answer
317 views

Newton optimization algorithm with non-positive definite Hessian

In the newton optimization algorithm to find the local minimum $x^*$ of a non-linear function $f(x)$ with iteration sequence of $x_0 \rightarrow x_1 \rightarrow x_2 ... \rightarrow x^*$ all $\nabla ^2 ...
3
votes
2answers
323 views

Solving for gradient of Frobenius norm term

Let's first define a couple of variables: $A,B,C \in \mathbb{R}^{m \times n}, D \in \mathbb{R}^{n \times n}$, and $\mu$ is a scalar. Say I have an ADMM sub-problem that looks like this: $\arg \min_{...
3
votes
3answers
63 views

Minimization of $log_{a}(bc)+log_{b}(ac)+log_{c}(ab)$?

I am trying to find the minimal value of the expression: $log_{a}(bc)+log_{b}(ac)+log_{c}(ab)$ I think experience gives that the variables should be equal, if so then the minimal value is 6, but ...
3
votes
2answers
3k views

Which optimization algorithm converges faster?

everyone. I'm having a large scale unconstrained optimization problem. If I treat the unconstrained problem as a constrained problem with infinity constraints, I should be able to use both the ...
3
votes
3answers
1k views

Constrained optimization: equality constraint

I have this very general problem (for $n>2$): $$ \begin{align} & \max Z = f(x_1,\ldots ,x_n) \\[10pt] \text{s.t. } & \sum_{i=1}^{n} x_i = B \\[10pt] & x_i \geq 0 \end{align} $$ Assume ...
3
votes
1answer
3k views

Prove the supremum of the set of affine functions is convex

Let $\langle f_i \rangle _{i \in I}$ be a family of affine functions on a convex and compact set $\Omega \subset \mathbb{R^d}$ such that $f_i = a_i.x +b_i$ for $x \in \Omega$. Prove that f, defined by ...
3
votes
2answers
185 views

Maximum of a product of a polynomial with positive coefficients and a finite sum of exponentials with negative coefficients on $[0,+\infty)$

Prove or disprove that $$ f(x)=\left(\sum_i a_i x^i\right)\left(\sum_j b_j e^{-\lambda_j x}\right) $$ where $\forall i, a_i>0$, $\forall j, b_j>0,\lambda_j>0$, and both sums are finite, ...
3
votes
2answers
65 views

The quadratic case in nonlinear programming

I'm reading about nonlinear programming and I stumbled into the following statement where I started to wonder a bit: Consider the function $$f(\textbf{x}) = \frac{1}{2}\textbf{x}^T\textbf{...
3
votes
1answer
912 views

Definition of tangent cone in continuous optimization .

Looking at the definition of tangent cone in continuous optimization : If $M$ is a open subset of $\mathbb R^n$ $x \in M$, The tangent cone of $M$ at $x$ is defined by $$\mathbb T (M, x) = \big\{d \...
3
votes
1answer
680 views

Optimizing trigonometric and nonlinear functions

First, Please, keep in mind that I'm a programmer not mathematician, and I have a fair mathematical background. I used optimization in Java to fit some observations to a trigonometric function, I ...
3
votes
1answer
75 views

Maximize $ 2^{(-x)} + 2^{(-y)}$ subjected to certain conditions

I am reading through convex optimization and I came across this following problem: \begin{align*} \max \text{ } & 2^{-x}+2^{-y}\\ \text{s.t. } & \frac{1}{1+x}+\frac{1}{1+y}\leq b\\ & x\...
3
votes
1answer
433 views

Levenberg's original article “A method for the solution of certain problems in least squares”

Does there exist any digital copy of the original article (or a transcript) K. Levenberg, A method for the solution of certain problems in least-squares, Quart. Appl. Math. 2 (1944): 164-168? It is (...
3
votes
2answers
255 views

Singular Values of Matrix as Optimization Problem

Assume that $A$ is a positive semidefinite symmetric matrix. It is known that $$\max_{||y||\leq1} \quad y^TAy$$ Has an analytical solution which is the maximum eigenvalue of $A$. This isn't hard ...
3
votes
1answer
2k views

Taylor's theorem for vector valued functions

I'm reading about linear and nonlinear programming and on one page I have the following statment (I have highlighted the areas where I have problems and drawn questions for them in the bottom of it): ...
3
votes
2answers
178 views

Optimization for constrained problem

I'm reading about Lagrange multipliers from a Pattern recognition book appendix and on one point the following is stated: $\begin{align} &\color{gray}{\verb+C.4+}\,\,\,\,\,\color{#08F}{\textbf{...
3
votes
1answer
130 views

minimizing $\sum_{i=1}^n \max(|x_i - x|, |y_i - y|)$

Suppose there are $n$ points $(x_i, y_i)$ for $i = 1,\ldots,n$. Please find another point $(x, y)$ to minimize function: $$\sum_{i=1}^n \max(|x_i - x|, |y_i - y|)$$
3
votes
1answer
843 views

Solving a system of non-linear (trig) equations:

I am having trouble trying to solve the following equations: $\sin(\alpha)+\sin(\beta)=\dfrac {1000} A$ $\sin(\alpha)+\sin(\gamma)=\dfrac {800} A$ $\dfrac {20(1+\cos(\alpha-\beta))} {\cos(\beta)} -\...
3
votes
1answer
271 views

Linear stability analysis on a constrained three-dimensional system of ODE

Let $\begin{cases} \dot x = f({\bf u}) \\ \dot y = g({\bf u}) \\ \dot z = h({\bf u})\end{cases}$ be a well-defined nonlinear system with ${\bf u} = (x,y,z)$ and restricted to domain $x,y,z \geq 0$. ...
3
votes
1answer
35 views

existence of solution of $Ax= \max(b-x,0) $

How do you prove the existence of a solution to the linear system: \begin{equation} Ax= \max(b-x,0) \end{equation} A is an $n\times n$ matrix and $b$ is a vector in $\mathbb{R}^n$. $x$ is the ...
3
votes
2answers
187 views

Upper bound for Maximization problem

I have an optimization problem of the form Max $x_1+x_2+x_3+\cdots+x_n$ subject to $x_0^2+x_1^2+x_2^2+\cdots+x_n^2+x_{12}^2+x_{13}^2+x_{14}^2+ \cdots+x_{1n}^2+x_{23}^2 + \cdots +x_{2n}^2+ \cdots +...
3
votes
1answer
213 views

non linear optimization

How to solve this optimization problem using matlab or some other tool. I know that, this is a convex problem with non-linear constraint $\rho\geq \rho_{min}$ , so i have tried many times it in ...
3
votes
2answers
2k views

Algorithm for GRG2 method of solving non-linear least square

I have been looking for quite a while for an algorithm for the GRG2 method either in a .net assembly or an algorithm i could program myself but I cant find a decent representation of the algorithm to ...
3
votes
1answer
313 views

Optimization with constraint on solution of a linear system

I'm facing this optimization problem: $$\text{minimize} \quad a^T x$$ $$\text{s.t. the solution of $A(x) z + B(x) = 0$ belongs to a convex set $S$}$$ Here $A(x)$ is a linear matrix function of $x$ ...
3
votes
1answer
32 views

Nonlinear integer programming problem

I am trying to maximize the following function $$ f(m,n) = \frac{m \log 3 + n \log 2}{\sqrt{m^2+n^2}} $$ where $ n $ and $ m $ are integers, not both $ = 0 $, although one could be $ 0 $. This is ...
3
votes
1answer
73 views

KKT conditions for a maximization problem

I have an optimization problem \begin{equation} \mathbf{w}^*= \text{argmax} \sum_{d=1}^{D}\log\left(\frac{|\mathbf{\hat{f}}_{d}^{H}\mathbf{w}|^{2}+A_d}{|\mathbf{\hat{f}}_{d}^{H}\mathbf{w}|^{2}+B_d}\...
3
votes
1answer
54 views

Find the critical point and show it is not a global minimizer (using Hessian)

Consider the function $f(x,y) = x^3 + e^{3y}-3xe^y$. Show that $f$ has exactly one critical point and that this point is a local minimizer, but not a global minimizer. I have attempted this, but it ...
3
votes
2answers
40 views

Distance between a plane and set of points

Suppose $m$ data points belonging to a class represented by matrix $A$. Therefore, the size of matrix $A$ is $m\times n$. In addition, suppose $w\cdot x + b=0$ be equation of a plane in $\mathbb{R}^n$....
3
votes
1answer
106 views

No clear analytic method to prove unique maximum? ($2^{-x}+2^{-1/x}$)

Prove that $f(x) = 2^{-x}+2^{-1/x}$ has the unique local maximum $(1,1)$ for $x>0$. Do not use computer software. Proving that $(1,1)$ is a maximum is easy, but I'm having trouble with the ...
3
votes
1answer
58 views

Non-linear system vs minimisation problem

If you have a non-linear system of equations which can be formally written as : \begin{equation} \begin{cases} F_1(\mathbf{x})=0\\ F_2(\mathbf{x})=0\\ \ \ \ \ \vdots\\ F_n(\mathbf{x})=0\\ \end{cases} ...
3
votes
1answer
492 views

fantasy basketball model

i'm creating a fantasy basketball model (could be used in other games too) where we can project how well a player will do against another team even when the player hasn't played against a certain team ...
3
votes
2answers
64 views

Sensitivity of polynomial global minimizers with respect to perturbations in the coefficients.

I'm trying to find the value of a global minimizers of a multivariate polynomial (4 variables) of high order numerically. The numerical values of the coefficients are coming from noisy measurements ...
3
votes
2answers
416 views

Maximizing “log det + log sum exp” function

I'm trying to find a numerical solution to the following optimization problem $$ \text{maximize } f(M) = \frac{1}{2} \log \det(M) + \log \sum_{i=1}^n \exp \left\{ - \frac{1}{2} x_i^T M x_i + a_i \...
3
votes
2answers
55 views

Max of $3$-Variable Function

I'm trying the find the maximum of the function $$f(a,b,c)=\frac{a+b+c-\sqrt{a^2+b^2+c^2}}{\sqrt{ab}+\sqrt{bc}+\sqrt{ca}}$$ for all nonnegative real numbers $a, b, c$ with $ab + bc + ca > 0$. I ...
3
votes
1answer
1k views

Lagrange Multipliers for Function Spaces

For some constant $A > 1$ I am trying to solve the constrained minimization problem minimize $F(u)$ in $C$ subject to $H(u) = 0$. Here $F(u) = \int -u dx$ and $H(u) = \int \sqrt{1 + (u')^2} dx - A$...