A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

1
vote
1answer
298 views

How do I convert a constraint with a product of two integer variables to a linear constraint?

I have a constraint of the form: $$\theta \leq a_1x_1 + a_2x_2 + a_3x_1x_2$$ where, $x_1$ and $x_2$ are integer variables with ranges $x_1 \in \{0, m\}$ and $x_2 \in \{0, n\}$. I would want to ...
1
vote
3answers
343 views

ADMM formalization

I found lots of examples of ADMM formalization of equality constraint problems (all with single constraint). I am wondering how to generalize it for multiple constraints with mix of equality and ...
1
vote
0answers
43 views

Do quasi-Newton methods check the second-order optimality condition?

I have a practical question about quasi-Newton methods. In quasi-Newton methods, Hessian matrix is approximated. It seems to be impossible for them to check the second optimality condition. In ...
0
votes
1answer
40 views

How to find optimized value of two variables

I have two variables: $\kappa_y$ and $\kappa_x$ And three functions: $M_y$=$M_y$($\kappa_y$, $\kappa_x$) $M_x$=$M_x$($\kappa_y$, $\kappa_x$) $F_z$=$F_z$($\kappa_y$, $\kappa_x$) All these three ...
2
votes
2answers
189 views

Why a convex cone cannot have more than one extreme point?

The way I define an extreme point is : A point which cannot be defined as a convex combination of two distinct points. I'm not able to extend this and show why a convex cone cannot have more than ...
1
vote
0answers
50 views

Why Was Backprop Invented?

I'm currently researching artificial neural networks and I keep wondering why do we use "backpropagation" to train a neural network. An ANN is basically just a very large and complex function $f(\...
1
vote
1answer
33 views

How can I compute fast the minimum of a linear plus Kulback-Leibler on the unit simplex?

Given $a, x^0 \in \mathbb{R}^n$ I wish to compute $$\min_{x \in \Delta_n} a^t x + \sum_{i=1}^n x_i\log(x_i/x^0_i) - x_i +x^0_i $$ where $\Delta_n$ is the unit simplex $\{x \in \mathbb{R}^n \mid \sum_{...
1
vote
1answer
48 views

Gauss-Newton Non-Linear Squares Optimisation

I doubt this is solvable at all, but I thought I will give a try. Essentially I am trying to extend Gauss-Newton algorithm to 2nd Taylor term. http://en.wikipedia.org/wiki/Levenberg–...
0
votes
1answer
27 views

How do I know that method of steepest descent works?

Here is the definition of the method of steepest descent given in the book "The mathematics of nonlinear programming" by Peressini. Suppose $f(x)$ is a function with continuous partial derivatives on ...
0
votes
0answers
37 views

Global stochastic maximization of a multi-parameter function

I have a function $F:\mathbb{R}^n\to[0,1]$ such that $$ F(\lambda) = \mathbb{E}_x[f(\lambda;x)] = \int f(\lambda;x)\mu(x)dx,$$ and I want to find $\tilde\lambda$ that maximizes F, i.e. $F(\tilde\...
4
votes
1answer
99 views

How to maximize $\sum\limits_{i=1}^n u_iln(x_i)$?

How to maximize this? $$ \sum\limits_{i=1}^n u_iln(x_i), $$ where $u_i,x_i$ are real numbers, $n$ is a positive integer, $0 \leq u_i \leq 1, 0 < x_i < 1, \sum\limits_{i=1}^n u_i = 1, \sum\...
3
votes
1answer
106 views

No clear analytic method to prove unique maximum? ($2^{-x}+2^{-1/x}$)

Prove that $f(x) = 2^{-x}+2^{-1/x}$ has the unique local maximum $(1,1)$ for $x>0$. Do not use computer software. Proving that $(1,1)$ is a maximum is easy, but I'm having trouble with the ...
1
vote
0answers
46 views

How to prove a solution is indeed a constrained minimum?

I'm reading the following example on Heath's Scientific Computing (page 266, second edition if anyone has it). "Minimize $f(x_1,x_2)=2\pi x_1(x_1+x_2)$ subject to $g(x_1,x_2)=\pi x^2_1x_2-V$" (...
1
vote
0answers
355 views

Epsilon constraint method - Pareto optimal solution representation

There's a course that I do remotely and I have a homework question which I have no idea how to answer. I did look up a lot in google and did not find any good examples - only loads of information and ...
1
vote
1answer
57 views

Non-uniqueness of worst-case (max-min or min-max) optimization

I have a worst-case optimization problem, where i want to maximize the minimum from the uncertainty set (uncertainty is given as an ensemble of 100 realizations, so an ensemble based approach). It is ...
0
votes
1answer
168 views

How do we plot nonlinear differential equations

If this is not nonlinear I apologize, I'm still learning differential equations. I am attempting to make a stream plot of a predator-prey model of eccentric closed curves by using the following ...
0
votes
0answers
42 views

Optimization on fixed sum

Consider this following scenario. Suppose I have $N$ cents, and I want to dispatch these money to $n$ people, each got $x_i$ cents. In order to simplify this problem, we assume the cents are "...
0
votes
1answer
28 views

How to optimize these parameter

How to optimize the following respect to lambda1 and lambda2: $\sum_{i} f(i) * log(\lambda_1 g(i) + \lambda_2 h(i))$ f(i), g(i), h(i) are known funtions Find lambda1 and lambda2 that satisfy $\...
0
votes
1answer
61 views

Necessary Conditions for Saddle Value point

This questions is from the Kuhn-Tucker paper "Nonlinear Programming" in Section 2 Lemma 1. I don't understand how those conditions are necessary for a saddle point. I always thought that a saddle ...
0
votes
1answer
61 views

Maximum likeliood estimation of variances of transformed variables

I use MATLAB's fminunc function in order to find the minimum of a negative log-likelihood function $f(\overrightarrow{\theta})$, parametrized by 3 parameters lets say $\overrightarrow{\theta}=(\beta,\...
0
votes
0answers
64 views

How can I minimize a quadratic on the unit simplex?

How can I compute $$ \min_{x \in \Delta_n} \frac{1}{2}\lVert Bx\rVert^2 + x^tAy$$ with $x \in \mathbb{R}^n, y \in \mathbb{R}^m, A_{m \times n}$, $B_{n \times n}$ where $\Delta_n$ is the unit simplex $$...
1
vote
0answers
67 views

Maximizing the frobenius norm subject to constraints $\underset{\mathbf{S}}{\text{maximize }} \|\mathbf{S}\|_F^2$

IF $\mathbf{X=AS}$ where $\mathbf{X} \in R_+^{n \times m}$, $\mathbf{A} \in R_+^{n \times r}$ are known variable and $\mathbf{S} \in R_+^{r \times m}$ is unknown variable, How to solve the below ...
0
votes
0answers
26 views

How can I solve $\min \{ \langle A(x),y\rangle + f(y) \text{ s.t. } y \in S^n, \operatorname{tr}(y) =1, y \geq 0\}$?

I'm trying to solve the problem $$\min \{ \langle A(x),y\rangle + f(y) \mid y \in S^m, \operatorname{tr}(y) =1, y \geq 0\}$$ where $x \in \mathbb{R}^n$, $y \in S^M$, that is, it's a symmetric $m$ by $...
1
vote
0answers
193 views

Global and local maxima in a weighted sum of logarithms of linear functionals?

Is is possible to describe, and locate efficiently, the maxima of the function below in the parameters $\mathbf{x}$ $$\sum_{i} p_i \log( N + \sum_j x_j[B_j +(A_j-B_j)\delta_{ij} + min(A_j,B_j) ]) \...
0
votes
2answers
91 views

Minimize a non-convex function subject to linear dynamics constraint

I want to solve the following problem: $$\min\limits_{\bf u} \frac{\bf c^T {\bf x} (T_f)}{\| \bf c\|\|{\bf x} (T_f)\|}$$ subject to $$\dot{\bf x} (t) = A {\bf x}(t) + B {\bf u}(t)$$ $$x(0) = x_0$$ ...
1
vote
2answers
67 views

Is the optimum of this problem unique?

I have a convex optimisation problem: $$\min_{x_i} \sum_{i=1}^N a_i \exp(-x_i/b_i)\quad\text{ s.t. }\sum_{i=1}^N x_i=x\text{ and } x_i\ge 0$$ The KKT conditions are: $$\lambda=\begin{cases} a_i/...
0
votes
1answer
687 views

Solving non-linear optimization using generalized reduced gradient (GRG) method

Consider the following elementary maximization problem: \begin{align} f{=}\mathrm{argmax}_{y_{l,c}, p_{l,c}}~\sum_{l=1}^{L}\sum_{c=1}^{C} y_{l,c}\text{log}_2\left(1+\frac{p_{l,c}}{I_{l,c}}\right) \end{...
1
vote
1answer
110 views

Minimize Product of Sums of Squared Distances

The Question Given two sets of vectors $S_1$ and $S_2$,we want to find a unit vector $s$ such that $$\{\sum_{u\in S_1}(\|u\|^2-\langle u, s \rangle^2)\} \cdot \{\sum_{v\in S_2}(\|v\|^2 - \langle v, ...
0
votes
1answer
56 views

Solving three-variable nonlinear equation systems

A physical problem which I've been studying leads to the following nonlinear equation system to be solved: $$\alpha\cdot79\cdot A_1 +(1-\alpha)\cdot 1025 \cdot B_1 = C_{11}$$ $$\alpha\cdot145\cdot A_1 ...
0
votes
2answers
102 views

How to find more than two coefficient for single variable nonlinear equation?

I don't have good knowledge on mathematics, but now I faced one problem with maths. That is, I have a data set which contains only one independent and one dependent variable. Now I have a equation ...
1
vote
1answer
151 views

how to write largest circle inscribed inside a triangle as an optimization problem?

can someone show me how to write this problem as a convex optimization problem.Find the largest disk that can be bounded by $X \geq 0$ , $Y \geq0$ and $X+2Y\leq1$. My institution is to cast to ...
1
vote
1answer
278 views

Minimize the squared dot product of two specific vectors

Do you think there exists a efficient algorithm(non brute-force) for the following problem. I search the optimal solution for the following problem: Given a vector $u=(u_1, u_2,..., u_k)^T$ with $\...
2
votes
1answer
206 views

Is standard eigenvalue optimization problem convex

For any arbitrary symmetric matrix A , is the standard eigenvalue problem convex $ \lambda_{max}(A)= \max_{\|x\| \leq1} x^{T}Ax$
1
vote
0answers
97 views

Tucker's theorem from Farkas lemma

I am trying to understand the proof of Tucker's theorem using Farkas lemma but there are some points that are not clear to me. The proof I am following is in this paper at page 16. What I do not ...
3
votes
1answer
58 views

Non-linear system vs minimisation problem

If you have a non-linear system of equations which can be formally written as : \begin{equation} \begin{cases} F_1(\mathbf{x})=0\\ F_2(\mathbf{x})=0\\ \ \ \ \ \vdots\\ F_n(\mathbf{x})=0\\ \end{cases} ...
0
votes
0answers
26 views

Broyden's Method mismatched dimensions

Compute the first two iterates $\mathbf{x}^{(1)}$, $\mathbf{x}^{(2)}$ using Broyden's Method for the initial point $\mathbf{x}^{(0)}=(1,4)$ and the function $f(x_{1},x_{2})=3x_{1}^2+x_{2}^2-x_1x_2$ ...
1
vote
2answers
62 views

Performing Second Derivative test on multivariate function

I have two functions $f=xy^2$ and $g=x^2+y^2$. When optimizing $xy^2$ on the circle $x^2+y^2=1$ I get 6 critical points but when I try to perform the second derivative test, it equals 0, meaning that ...
1
vote
0answers
130 views

what is the good book to learn nonlinear programming

I would like to learn nonlinear programing. what is the best book to do so and I prefer if the solution manual of the book is available. thanks
1
vote
1answer
152 views

Optimization over vector spaces. Generalized KKT.

I am looking for the extension of the theorem I found in the book by Luenberger called "Optimization by vector space methods." Here is the statement of that theorem from Luenberger: Generalized ...
0
votes
1answer
93 views

Calculating the Hessian for nonlinear objective functions

In least squares minimization, we minimize the sum of squares of the residuals: $f = ||b-Ax||_2 ^2 $ The Hessian of this matrix is given by $2A^TA$, and from the Hessian we can do the second partial ...
1
vote
0answers
69 views

How to solve coupled nonlinear ODEs with a algebraic constraints?

Here is the problem I'm currently facing right now, I have set of ODEs which I can solve numerically given the initial condition. But I'm not sure how to go about giving algebraic solution with ...
1
vote
1answer
38 views

Prove a specific Cartan matrix is positive definite

I am trying to prove that the following matrix is positive definite, but I am stuck in the last step of my proof... Any help would be really appreciated. Thanks! Question Let $A$ be a matrix with ...
0
votes
1answer
60 views

Can a symmetric, positive-definite, real matrix with only 1s on the main diagonal have an off diagonal element with absolute value greater than 1?

I am working with correlation matrices and I would like to know if every symmetric, positive definite matrix with 1s on the main diagonal is a correlation matrix (i.e. all its off diagonal elements ...
0
votes
0answers
179 views

Optimizing concave function over non-convex set

I have the following problem that I am looking advice on. Let $ \mathcal{F}$ be a convex subset of vector space $X$. The goal it to \begin{align*} \max_{x \in \mathcal{F}} f(x)\\ s.t. \ g(x) \le 0 \...
0
votes
2answers
85 views

Explaining the “well-known” optimization of this particularly simple convex, non-differentiable function?

I've been programming algorithms for solving L1-regularized logistic regression with large datasets. As such, I've been delving into the computer science literature, and came across the following ...
0
votes
2answers
2k views

What is the merit function?

When we use merit function in optimization & why uses this function? if we use merit function the space must be convex or not?
0
votes
1answer
79 views

How to solve this problem efficiently?

I have this problem \begin{align} \min_{\alpha,\beta,X}~&<\alpha \cdot X+\beta \cdot Y,D>-c \cdot (<\alpha \cdot X+\beta \cdot Y,H>)^{1/2}\\ &X,\alpha,\beta>=0\ \end{...
0
votes
1answer
56 views

what is the trust region algorithm in optimization?

I see some books that say the trust region work with contour's line .but i can't understand how choose the point with contour's line and sort them? thank you if answer me.
1
vote
0answers
76 views

Basis pursuit denoising

The Lagrangian form of basis pursuit denoising $\min_{w} ||w||_{\ell_{1}} + \lambda ||Aw-x||_{\ell_{2}}^{2}$ can be solved using proximal gradient descent. Proximal methods also can be used to ...
2
votes
1answer
22 views

Showing that $x_{k+2}$ is a point which approximates a maximum?

Suppose that $x_k<x_{k+1}$ and $f'(x_k)>f'(x_{k+1})$. How can I show that the secant method will give $x_{k+2}$ as a point which approximates a maximum? $$x_{k+2}= x_k-\frac{f'(x_k)(x_{k+1}-...