A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

3
votes
3answers
116 views

How to find the root of a polynomial function closest to the initial guess?

I need some easy to implement and fast numerical method that finds the root of a nonlinear function (a polynomial in my case) closest to my initial guess. If I know that there is one root ...
3
votes
2answers
60 views

$g$ is coercive for $g(x)=x^TAx+b^Tx+c$

Suppose $A$ is a symmetric positive definite matrix $A\in \Bbb{R}^{n\times n}$, $b \in \Bbb{R}^n$, and c is a real number. Let $$g(x)=x^TAx + b^Tx + c$$ Show that $g$ is coercive. Because $A$ is ...
3
votes
3answers
63 views

Minimization of $log_{a}(bc)+log_{b}(ac)+log_{c}(ab)$?

I am trying to find the minimal value of the expression: $log_{a}(bc)+log_{b}(ac)+log_{c}(ab)$ I think experience gives that the variables should be equal, if so then the minimal value is 6, but ...
3
votes
1answer
201 views

Newton optimization algorithm with non-positive definite Hessian

In the newton optimization algorithm to find the local minimum $x^*$ of a non-linear function $f(x)$ with iteration sequence of $x_0 \rightarrow x_1 \rightarrow x_2 ... \rightarrow x^*$ all $\nabla ^2 ...
3
votes
2answers
197 views

Solving for gradient of Frobenius norm term

Let's first define a couple of variables: $A,B,C \in \mathbb{R}^{m \times n}, D \in \mathbb{R}^{n \times n}$, and $\mu$ is a scalar. Say I have an ADMM sub-problem that looks like this: $\arg ...
3
votes
1answer
3k views

Prove the supremum of the set of affine functions is convex

Let $\langle f_i \rangle _{i \in I}$ be a family of affine functions on a convex and compact set $\Omega \subset \mathbb{R^d}$ such that $f_i = a_i.x +b_i$ for $x \in \Omega$. Prove that f, defined by ...
3
votes
3answers
1k views

Constrained optimization: equality constraint

I have this very general problem (for $n>2$): $$ \begin{align} & \max Z = f(x_1,\ldots ,x_n) \\[10pt] \text{s.t. } & \sum_{i=1}^{n} x_i = B \\[10pt] & x_i \geq 0 \end{align} $$ Assume ...
3
votes
2answers
179 views

Maximum of a product of a polynomial with positive coefficients and a finite sum of exponentials with negative coefficients on $[0,+\infty)$

Prove or disprove that $$ f(x)=\left(\sum_i a_i x^i\right)\left(\sum_j b_j e^{-\lambda_j x}\right) $$ where $\forall i, a_i>0$, $\forall j, b_j>0,\lambda_j>0$, and both sums are finite, ...
3
votes
2answers
62 views

The quadratic case in nonlinear programming

I'm reading about nonlinear programming and I stumbled into the following statement where I started to wonder a bit: Consider the function $$f(\textbf{x}) = ...
3
votes
1answer
736 views

Definition of tangent cone in continuous optimization .

Looking at the definition of tangent cone in continuous optimization : If $M$ is a open subset of $\mathbb R^n$ $x \in M$, The tangent cone of $M$ at $x$ is defined by $$\mathbb T (M, x) = \big\{d ...
3
votes
1answer
585 views

Optimizing trigonometric and nonlinear functions

First, Please, keep in mind that I'm a programmer not mathematician, and I have a fair mathematical background. I used optimization in Java to fit some observations to a trigonometric function, I ...
3
votes
2answers
53 views

Turn off the ovens! An optimization problem

The problem is more abstract, but can be illustrated nicely using ovens. A oven can produce any heat, but is most efficient when it produces $c$ heat. The inefficency increases quadratically as one ...
3
votes
1answer
72 views

Maximize $ 2^{(-x)} + 2^{(-y)}$ subjected to certain conditions

I am reading through convex optimization and I came across this following problem: \begin{align*} \max \text{ } & 2^{-x}+2^{-y}\\ \text{s.t. } & \frac{1}{1+x}+\frac{1}{1+y}\leq b\\ & ...
3
votes
2answers
65 views

How does one choose the step size for steepest descent?

Consider finding the minimal value for any function $g$ from $\mathbb{R}^n$ to $\mathbb{R}$. The method of steepest descent for finding a local minimum for an arbitrary function $g$ from from ...
3
votes
1answer
2k views

Intuition for gradient descent with Nesterov momentum

A clear article on Nesterov’s Accelerated Gradient Descent (S. Bubeck, April 2013) says The intuition behind the algorithm is quite difficult to grasp, and unfortunately the analysis will not be ...
3
votes
2answers
218 views

Singular Values of Matrix as Optimization Problem

Assume that $A$ is a positive semidefinite symmetric matrix. It is known that $$\max_{||y||\leq1} \quad y^TAy$$ Has an analytical solution which is the maximum eigenvalue of $A$. This isn't hard ...
3
votes
1answer
2k views

Taylor's theorem for vector valued functions

I'm reading about linear and nonlinear programming and on one page I have the following statment (I have highlighted the areas where I have problems and drawn questions for them in the bottom of it): ...
3
votes
2answers
173 views

Optimization for constrained problem

I'm reading about Lagrange multipliers from a Pattern recognition book appendix and on one point the following is stated: $\begin{align} ...
3
votes
1answer
124 views

minimizing $\sum_{i=1}^n \max(|x_i - x|, |y_i - y|)$

Suppose there are $n$ points $(x_i, y_i)$ for $i = 1,\ldots,n$. Please find another point $(x, y)$ to minimize function: $$\sum_{i=1}^n \max(|x_i - x|, |y_i - y|)$$
3
votes
1answer
726 views

Solving a system of non-linear (trig) equations:

I am having trouble trying to solve the following equations: $\sin(\alpha)+\sin(\beta)=\dfrac {1000} A$ $\sin(\alpha)+\sin(\gamma)=\dfrac {800} A$ $\dfrac {20(1+\cos(\alpha-\beta))} {\cos(\beta)} ...
3
votes
1answer
242 views

Linear stability analysis on a constrained three-dimensional system of ODE

Let $\begin{cases} \dot x = f({\bf u}) \\ \dot y = g({\bf u}) \\ \dot z = h({\bf u})\end{cases}$ be a well-defined nonlinear system with ${\bf u} = (x,y,z)$ and restricted to domain $x,y,z \geq 0$. ...
3
votes
1answer
34 views

existence of solution of $Ax= \max(b-x,0) $

How do you prove the existence of a solution to the linear system: \begin{equation} Ax= \max(b-x,0) \end{equation} A is an $n\times n$ matrix and $b$ is a vector in $\mathbb{R}^n$. $x$ is the ...
3
votes
2answers
183 views

Upper bound for Maximization problem

I have an optimization problem of the form Max $x_1+x_2+x_3+\cdots+x_n$ subject to $x_0^2+x_1^2+x_2^2+\cdots+x_n^2+x_{12}^2+x_{13}^2+x_{14}^2+ \cdots+x_{1n}^2+x_{23}^2 + \cdots +x_{2n}^2+ \cdots ...
3
votes
1answer
188 views

non linear optimization

How to solve this optimization problem using matlab or some other tool. I know that, this is a convex problem with non-linear constraint $\rho\geq \rho_{min}$ , so i have tried many times it in ...
3
votes
2answers
2k views

Algorithm for GRG2 method of solving non-linear least square

I have been looking for quite a while for an algorithm for the GRG2 method either in a .net assembly or an algorithm i could program myself but I cant find a decent representation of the algorithm to ...
3
votes
1answer
300 views

Optimization with constraint on solution of a linear system

I'm facing this optimization problem: $$\text{minimize} \quad a^T x$$ $$\text{s.t. the solution of $A(x) z + B(x) = 0$ belongs to a convex set $S$}$$ Here $A(x)$ is a linear matrix function of $x$ ...
3
votes
1answer
63 views

KKT conditions for a maximization problem

I have an optimization problem \begin{equation} \mathbf{w}^*= \text{argmax} ...
3
votes
1answer
39 views

Find the critical point and show it is not a global minimizer (using Hessian)

Consider the function $f(x,y) = x^3 + e^{3y}-3xe^y$. Show that $f$ has exactly one critical point and that this point is a local minimizer, but not a global minimizer. I have attempted this, but ...
3
votes
2answers
39 views

Distance between a plane and set of points

Suppose $m$ data points belonging to a class represented by matrix $A$. Therefore, the size of matrix $A$ is $m\times n$. In addition, suppose $w\cdot x + b=0$ be equation of a plane in ...
3
votes
1answer
89 views

No clear analytic method to prove unique maximum? ($2^{-x}+2^{-1/x}$)

Prove that $f(x) = 2^{-x}+2^{-1/x}$ has the unique local maximum $(1,1)$ for $x>0$. Do not use computer software. Proving that $(1,1)$ is a maximum is easy, but I'm having trouble with the ...
3
votes
1answer
55 views

Non-linear system vs minimisation problem

If you have a non-linear system of equations which can be formally written as : \begin{equation} \begin{cases} F_1(\mathbf{x})=0\\ F_2(\mathbf{x})=0\\ \ \ \ \ \vdots\\ F_n(\mathbf{x})=0\\ \end{cases} ...
3
votes
1answer
408 views

fantasy basketball model

i'm creating a fantasy basketball model (could be used in other games too) where we can project how well a player will do against another team even when the player hasn't played against a certain team ...
3
votes
2answers
62 views

Sensitivity of polynomial global minimizers with respect to perturbations in the coefficients.

I'm trying to find the value of a global minimizers of a multivariate polynomial (4 variables) of high order numerically. The numerical values of the coefficients are coming from noisy measurements ...
3
votes
2answers
365 views

Maximizing “log det + log sum exp” function

I'm trying to find a numerical solution to the following optimization problem $$ \text{maximize } f(M) = \frac{1}{2} \log \det(M) + \log \sum_{i=1}^n \exp \left\{ - \frac{1}{2} x_i^T M x_i + a_i ...
3
votes
2answers
54 views

Max of $3$-Variable Function

I'm trying the find the maximum of the function $$f(a,b,c)=\frac{a+b+c-\sqrt{a^2+b^2+c^2}}{\sqrt{ab}+\sqrt{bc}+\sqrt{ca}}$$ for all nonnegative real numbers $a, b, c$ with $ab + bc + ca > 0$. I ...
3
votes
1answer
958 views

Lagrange Multipliers for Function Spaces

For some constant $A > 1$ I am trying to solve the constrained minimization problem minimize $F(u)$ in $C$ subject to $H(u) = 0$. Here $F(u) = \int -u dx$ and $H(u) = \int \sqrt{1 + (u')^2} dx - ...
3
votes
1answer
632 views

Lagrangian Multipliers

I have a fundamental question about Lagrange multipliers. Here it is: I have a function to maximize with respect to a parameter say $\theta$, subject to two constraints. Lets assume that the first ...
3
votes
3answers
298 views

simple-looking non-convex optimization problem

I want to solve the following problem: Maximize $\sum_{i=1}^n\log(1+\lambda_i^2)$ subject to $\lambda_i >0$ and $\sum_{i=1}^n\lambda_i = M$. I was wondering how I could cast it as a convex problem. ...
3
votes
1answer
301 views

Finding a function that satisfies constraints numerically

I have the following system of equations for function $p(y)$ and I need help debugging my solution: $$\begin{align} 0&=\log(p(y))+1-\lambda-\gamma y^2-\eta ...
3
votes
1answer
317 views

solving linear program with rank constraint?

I have a linear program where the variables are n vectors. Now I'd like to impose an extra constraint that k (k<=n) of the n vectors are linearly independent, or the matrix with the n vectors as ...
3
votes
0answers
32 views

How to use the Karush–Kuhn–Tucker conditions?

From what I read, the Karush-Kuhn-Tucker conditions are a generalization of the Lagrange Multiplier Method. For the Lagrange Multiplier Method I have been able to find a serie of steps I must do to ...
3
votes
0answers
98 views

If this problem is not unbounded, what's wrong in this dual derivation?

In a paper with 100 citation, Robust Support Vector Machine Training via Convex outlier Ablation, a convex relaxation is used. In this paper, a form of robust svm proposed: \begin{align} \min_{0\leq ...
3
votes
1answer
77 views

Finding the root for a multivariate function.

Background In a practical problem I need to find the solution to: $$f(\bar{x}) - \bar{p} = \bar{0}$$ where $ f : \mathbb{R}^2 \rightarrow \mathbb{R}^2 $. I don't know the exact expression for $f$ ...
3
votes
0answers
64 views

[Levenberg-Marquardt]What is the link between positive-definiteness and well-conditioning?

Working on optimization problems through neural networks, I use the Levenberg-Marquardt algorithm. I have read this assertion that I do not understand : A positive definite diagonal matrix is ...
3
votes
0answers
153 views

Determining active constraints in KKT

Suppose there is a constrained optimization problem having inequality constraints. We can solve it using Karush-Kuhn-Tucker conditions. My question is how do we determine which constraints are active ...
3
votes
0answers
41 views

Solving an inverse squared sum

How would I go about solving this sum for $x$? $$\sum_i\frac{a_i}{(x+b_i)^2}=C$$ Where $\mathbf{a}$ and $\mathbf{b}$ are vectors and $C$ is a constant, and $x$ is a single number. It's for an ...
3
votes
0answers
60 views

Structural / design / meta optimization - is there mathematical theory. Optimization over categories?

There is huge branch of mathematical optimization theory, but it mostly considers the finding optimal parameter values for the predefined structures. There are variational calculus and optimal control ...
3
votes
0answers
194 views

Finding optimal velocity profile using Dynamic Programming

This question has been asked on scicomp but I thought maybe it's more a mathematical problem of how Bellman's idea is to be applied here. The main problem for me is: How to introduce the time ...
3
votes
0answers
42 views

Solve non-linear equation with Matrices

I'm looking more for hints than specific answers, although I would be extremely grateful if provided with one. The problem I have is as follows: $$ -\Sigma (A+\Lambda_1)+I=0 $$ Here A is a constant, ...
3
votes
1answer
85 views

Strong duality: When does the optimal primal variable coincide with the primal variable giving the dual function.

I'm considering the inequality-constrained optimization problem of finding $$ x^{\star} = \arg \min_{x} f(x) \;\; \text{s.t.} \;\; h(x) \le 0 $$ which is assumed to have a unique minimizer. The ...