A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

0
votes
0answers
10 views

Are there any optimization strategy suiting this framework?

For optimization problem: $min \quad f(x_1, x_2)$ Are there some strategies that are doing this sequentially, i.e. first solve $min_{x_1} \quad f(x_1, x_2^{k})$ to get $x_1^{k+1}$ then solve $min_{...
1
vote
2answers
47 views

Optimization over min function

We want to maximize $\sum_{i=1}^N \sum_{j=1}^N \min(a_i,b_j)$ such that $\sum_{i=1}^N a_i =1$ and $\sum_{j=1}^N b_j =1$. I think the optimal solution is $a_i = 1/N$ and $b_j = 1/N$ for $i,j = {1,2, ......
0
votes
1answer
46 views

Identify if optimization problem is convex or non-convex?

I have formulated optimization problem for building, where cost concerns with energy consumption and constraints are related to hardware limits and model of building. To solve this formulation, I need ...
0
votes
0answers
30 views

what should i replace the product of nonlinear variables with when linearizing

I have a product of two continuous variables in a constraint of an optimization problem. I want to linearize the product and use $$x_1 \cdot x_2 = y_1^2 - y_2^2$$ I followed the steps mentioned in ...
0
votes
1answer
85 views

Is this illustration of Gauss Newton wrong?

In this illustration the value of each iteration is the minimum of the 2nd derivative. But the Wikipedia page says: the advantage [of the Gauss–Newton algorithm] is second derivatives, which ...
0
votes
0answers
15 views

Preconditioning vs Re-orthogonalise for Non-Linear Conjugate Gradient Method

I am using CG to solve an optimisation problem. Since my cost function is ill-conditioned, I am looking at improving the performance either by using Preconditioning or Re-orthogonalisation, especially ...
0
votes
0answers
26 views

Using least squares regression to apply nonlinear function to time series data

If you have a nonlinear function (see example), can you use a least squares regression approach to fit it to time series data ? Is this approach also valid for n variables? How many time points are ...
0
votes
0answers
31 views

KKT conditions for nonlinear problem

I need to state the KKT conditions for the following problem: Minimise $x_1^2 + 2x_2^2$ subject to $(x_1-1)^2 + x_2^2 \le 1$ and $x_2 = 1$. I have that these conditions are: $f(x^*) \le 0$ $h(x^*)...
1
vote
1answer
15 views

Feasible set and level sets

Consider the following problem: Minimise $x_1^2 + 2x_2^2$ subject to $(x_1)^2 + x_2^2 \le 1$ and $x_2 = 1$. Sketch the feasible set and the level sets of the objective function, and determine an ...
1
vote
2answers
44 views

How to introduce Levenberg-Marquardt?

How would you introduce the Levenberg-Marquardt algorithm: To someone who understand the concept of minimisation and derivative. By using intuition instead of equation if possible. For instance a ...
0
votes
1answer
34 views

Does the non-expansion property of the projection operator hold for all definitions of norm?

For convex problem, of course. I vaguely remember this holds for weighted norm also. But I am curious if there are some general conclusions about what kinds of norm will fit in this framework?
1
vote
1answer
27 views

Minimizing a quadratic function of 2 variables in quadratic region

Let $f$ be a real valued quadratic function of 2 real variables: $$f(x,y) = ax^2 + by^2 + cxy + dx + ey + f$$ How to minimize it? Subject to constraints: $$ 0\leq x \leq 1, \quad 0\leq y \leq 1 $$ ...
1
vote
0answers
43 views

Scaled proximal operator for proximal Newton method

The scaled proximal operator was introduced as an extension of the (regular) proximal operator: $prox^H_h(x) = \arg\min_y h(y) + \frac{1}{2}\|y-x\|^2_H$. (See http://papers.nips.cc/paper/4740-...
0
votes
0answers
28 views

How to solve the following convex constrained optimization problem?

\begin{equation}\label{constrained optimization} \begin{aligned} \min\limits_{\mathbf{X}}&\|X_{(1)}\|_{*}+\|X_{(2)}\|_{*}+\|X_{(3)}\|_{*}+\lambda\|Ax-b\|_2^2 &\ \ s.t. X_{ijk}=M_{ijk}\ \ \...
0
votes
0answers
35 views

Minimization/maximization of system of nonlinear equations

Consider a system of nonlinear equations of the following form: $$F_1(x_2, x_3, x_4...x_n)$$ $$F_2(x_1, x_3, x_4...x_n)$$ $$...$$ $$F_n(x_1, x_2, x_3...x_{n-1})$$ And we wish to simultaneously ...
0
votes
0answers
27 views

nonlinear optimization : restricting search space to “limited” preselected value

My function is nonlinear with respect to a scalar \alpha . However, the calculation of objective function is very time consuming, making optimization also very time consuming. Also, I have to do it ...
1
vote
1answer
114 views

Dynamic programming at a non linear programming problem

Could you explain to me how we can use dynamic programming in order to solve a non linear programming problem? What do we do for example if we are given the following problem? $$\max (y_1^3-11 y_1^2+...
0
votes
0answers
27 views

minmizing frobenius norm($\min_{K \in \mathbb C^{n \times m}} \|KQK^*\|_F$) subject to an equality

In which known approach or algorithm a fat matrix $K\in \mathbb C^{n \times m} ,m>n,$ can be found: $\min_{K \in \mathbb C^{n \times m}} \|KQK^*\|_F$ subject to: $KK^*=I$ $Q\in \mathbb C^{m \...
0
votes
0answers
30 views

Minimizing summation using Karush–Kuhn–Tucker

Let $a_j, c_j , j=1,...n$ and b be positive constants. $$-$$ Minimize : $\sum_{j=1}^n \frac{c_j}{x_j}$ $$-$$ subject to:$\sum_{j=1}^n a_j x_j =b$ $$-$$ Write down the kkt conditions and solve the ...
0
votes
0answers
34 views

Optimization by KKT-method

I need to solve the following problem by KKT method. $$ \text{min} \ \ 2xy + 2yz + 2zx \\ \text{subject to} \ x^2 + y^2 ≤ 2, \ 2x + 2y + z = 0 $$ I have gotten as far as setting up the system of ...
0
votes
0answers
15 views

Using sequence of least squares solutions to solve non-linear problems?

I do know about the iteratively reweighted least-squares and have played around with it to some success finding non-linear solutions (like minimizing non-2-norms to achieve solutions which seem to be ...
0
votes
0answers
16 views

What is the moment matrix in converting polynomial optimization problem to a quadratic optimization problem

Happy new year, I have a function of the form below \begin{align} f(x,y,z)=\sum_i x_i y_i z_i + g(x)+h(z)\cr x,y,z \in R^n \end{align} where $h,g$ are quadratic functions. My difficulty lies in the ...
0
votes
1answer
53 views

Linear Optimization Problem with exponential variable

Hey Folks I've encountered an optimization problem which has a linear programming structure but it's coefficients are nonlinear function of another variable. here is the problem: $$\max _{{p_i},\...
0
votes
2answers
73 views

How to efficently solve a convex optimization problem with positive semi-definite Hessian matrix?

Consider the following optimization: $$ f(x)= \min \sum_{i=1}^n \left(x_i-\sum_{j=1}^n x_j\right)^2 $$ Let $g_i(x)=x_i-\sum_{j=1}^n x_j$ , then $$ f(x)= \min \sum_{i=1}^n g_i(x)^2 $$ The ...
1
vote
1answer
50 views

Positive-Semi-Definite form of Variance?

first thing: I'm an informatics student and know some algebra. However, this seems to be a bit over my head, so please be gentle with me. ;) I have multiple sets of real variables. Let these sets be $...
3
votes
1answer
75 views

Maximize $ 2^{(-x)} + 2^{(-y)}$ subjected to certain conditions

I am reading through convex optimization and I came across this following problem: \begin{align*} \max \text{ } & 2^{-x}+2^{-y}\\ \text{s.t. } & \frac{1}{1+x}+\frac{1}{1+y}\leq b\\ & x\...
4
votes
1answer
203 views

Monotonic Function Optimization on Convex Constraint Region

So I have the following function, which I want to maximize: $$f(x_1,...,x_n) = \sum_{i=1}^n\alpha_i\sqrt{x_i}$$ (where all $\alpha_i$ are positive), subjected to the following equality and inequality ...
0
votes
0answers
22 views

finding the least absolute inner product solution for some give vectors

Are there any efficient algorithms (like for PCA) that can find the orthogonal vector of a given bunch of vectors? Mathematically, let $\mathbf{b}_1, \ldots,\mathbf{b}_m\sim\mathcal{N}(\mathbf{0},\...
0
votes
0answers
27 views

Is there any approach towards finding all infinite solutions of a set of nonlinear equations when the number of unknowns is more than equations?

I have 3 nonlinear equations with 4 unknowns, with some bound constraints. How can I see if there is a solution to the problem? I wonder if there is a similar approach in nonlinear equations like SVD  ...
0
votes
0answers
52 views

Maximizing Frobenius norm

I was wondering if anybody has any suggestions on the following problem: Let S be an $n \times n$ real symmetric matrix and $W$ is a real matrix of size $n\times d$; $1\leq d <n$. $$ \text{Find ...
4
votes
0answers
81 views

How to use the Karush–Kuhn–Tucker conditions?

From what I read, the Karush-Kuhn-Tucker conditions are a generalization of the Lagrange Multiplier Method. For the Lagrange Multiplier Method I have been able to find a serie of steps I must do to ...
2
votes
1answer
31 views

proving that a global maximizer does not exist

Show that no matter what the value of $a$ is chosen, the function $f(x_1,x_2)=x_1^3-3ax_1x_2+x_2^3$ has no global maximizers. Determine the nature of the critical points of this function for all ...
1
vote
0answers
29 views

Proving that a function is coercive

Let $f(x,y)=x^2-2xy+y^2$. I know this is not coercive as along the line $y=x$, when $||x|| ->\infty, f(x,x)=0$. But I don't understand what is wrong with the following way. $$f(x,y)=x^2-2xy+y^2=...
0
votes
1answer
36 views

Is $|x^3|$ convex?

Let $f(x)=|x^3|$ on I=$-\infty,+\infty$ Is this convex? How I did was f(x) = \begin{cases} x^3, & \text{if $x>=0$ } \\ -x^3, & \text{if $x<0$ } \end{cases} Then$ f '(x)$ = \...
0
votes
0answers
44 views

How to Prioritize Constraints of an optimization problem?

I have an optimal control problem of a vehicle the form: $$ \min_{u \in R} \ell (x(j),u(j)) = {\Vert x(j)- x_r(j)\Vert}_Q + {\Vert u(j)- u_r(j)\Vert}_R $$ subject to $$\dot{x}=f(x)$$ $$g(x)>b$...
1
vote
2answers
44 views

Are these two optimization problems equal?

The first optimization model: $$ \begin{array}{cl} \arg \min \limits_{C} & \sum\limits_{i=1}^{3}\gamma_i\|{C_{(i)}}\|_*\\ \mathrm{s.t.} & \|A\mathbf{c}-\mathbf{b}\|_2^2+ \mathbf{c}^TG\...
2
votes
0answers
19 views

Fitting nonlinear differential equations to correspond a predefined solution

When modeling temporal dynamics of a biological process I stumbled upon a set of differential equations having the following matrix form: $$\frac{d\mathbf{a}(t)}{dt}=\Gamma(t)\mathbf{a}(t)+C\mathbf{a}...
1
vote
0answers
56 views

Levenberg-Marquart with Hessian and gradient

I am minimizing a sum-of-squared-differences function using the Levenberg-Marquardt method. The off-the-shelf numerical implementations I have have looked through (MATLAB, Numerical Recipes in C (15.5)...
0
votes
0answers
3 views

Leveraging the inverse in nonlinear optimization

Consider a non-linear optimisation problem like $$\mathcal{L} = \left\|{\bf x} - f({\bf y})\right\|$$ which we aim to minimise for vector ${\bf y}$ and where $f(.) : \mathbb{R}^N \mapsto \mathbb{R}^...
0
votes
0answers
18 views

How find the roots of non-convex function?

How to find a roots of non-convex function f(x)=0, where f is real scalar function of real scalar argument. What methods are exist for it? Or/And where I can to read about it?
0
votes
1answer
33 views

Minimizing a quadratic function with constraints on some variables

Consider a problem with strictly convex quadratic objective with some of the unconstrained variables. minimize: $x_1^TP_{11}x_1 + 2x_1^TP_{12}x_2 + x_2^TP_{22}x_2$ subject to: $f_i(x_1) \leq0, i = 1,...
2
votes
1answer
103 views

How to find the minimum distance from a point to a set?

Let $M=\{x: x_{1}^{2}+x_{2}^{2}+x_{3}^{2}\le4, x_{1}^{2}-4x_{2}\le0\}$ and $y=(1,0,2)^{T}$. Find the minimum distance from $y$ to $M$, the unique minimizing point and a separating plane. Does anyone ...
0
votes
0answers
38 views

How to solve the following optimization problem with projection?

How to solve the following optimization problem with projection? \begin{alignat}{1} &\min_{u_+,u_-,s,l\geq 0} \frac{1}{\lambda} \langle A ,(a +u_+-u_-)(a +u_+-u_-)^\mathsf{T} \rangle+\mathbf{1}^\...
1
vote
1answer
46 views

Simultaneous diagonalization of two symmetric matrices where one is pd

Let $A,B,C$ be two symmetric $n$ x $n$ matrices, where $B$ is also positive definite. Imitate the procedure to obtain a spectral decomposition of $A$ with respect to $B$, by replacing the constraints $...
1
vote
0answers
26 views

Examples of complex analysis useful in optimization?

Are there any examples of complex analysis applicable to mathematical optimization problems (preferably non-linear optimization)? I am wondering what advantages the use of complex numbers would have ...
2
votes
1answer
43 views

How to solve this KKT problem?

Given an optimization as follows: \begin{align} \text{minimize}\quad &c^Tx \\ \text{subject to}\quad &Ax = 0 \\ & \|x\|_2^2 \leq 1 \end{align} where $A \in \Re^{m\times n}$ is of ...
1
vote
0answers
25 views

What class of problem is a set of equations using inequalities and if-then-else?

Can you please identify what class of problem this is so that I can research algorithms for solving it please? Its a a set of linear equations and inequalities/constraints looking like this: ...
2
votes
1answer
33 views

Finding the gradient in least squares

In Linear squares optimization I have A=\begin{pmatrix} 1 & t_1 & t_1^2 & \cdots & t_1^k \\ 1 & t_2 & t_2^2 & \cdots & t_2^k \\ \vdots & \vdots& ...
0
votes
0answers
25 views

Making a distribution equally up to a maximum … but with a minimum!

So this seems to me like a linear programming problem, but I am getting some odd results. Forgive me, I'm not going to get the terminology correct, I know, so I will present the problem in simple ...
-1
votes
1answer
37 views

KKT Conditions for NLP [closed]

How may I state the KKT conditions for minimize $f(x) = ax^2$ subject to $Ax \leq b$, $x$ unrestricted?