Questions tagged [nonlinear-optimization]

A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for trying to solve particular problems.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Uniqueness in minimisation of Lp-Norm using Gradient Descent

I am trying to estimate the path of a random described by the following SSM ...
user avatar
0 votes
0 answers
6 views

Gradient of an objective function containg coupled odes

I have position values of the form $\mathbf{X} = \left[x \; \;y \right]^{T}$. Frenet-Serret model for 2D would consist of following equations: $$ \frac{d{\mathbf{X}}_{model}}{dt} = V(t){\mathbf{T}} $$ ...
user avatar
0 votes
0 answers
12 views

Constrained optimization of quadratic min function

Say I have an objective function $f(\mathbf{x}) = \min ( x_1^2, a_1 ) + \min ( x_2^2, a_2 )$ that I want to maximise subject to constrains $x_1 \geq b_1$ and $x_2 \geq b_2$. This obviously isn't ...
user avatar
  • 67
0 votes
1 answer
12 views

Split Bregman Algorithm for L1 optimization

I know that split bregman algorithm can be used for $L1$ norm optimization problem. In literature I have seen solving the problem of $x =: \underset{x}{\text{argmin }}\frac{1}{2}||y-Ax||^2+||x||_1$ ...
user avatar
1 vote
0 answers
24 views

How is the Wilson-Han-Powell SQP algorithm applied?

Say for example we need to minimize $x_2$ subject to $x_1^2+x_2^2-1=0$ starting at $x_1=x_2=1/2$ and using $B=\nabla^2[x_2+\lambda(x_1^2+x_2^2-1)]$ with $\lambda=1$. Now, the WHP-SQP algorithm goes ...
user avatar
0 votes
2 answers
66 views

Proof that markov chain equilibrium using Farkas' lemma

Given a transition matrix for markov chain $ P \in \mathbb R^{dxd} $ such that $$ P_{i,j} \geq 0,\quad 1 \leq (i,j) \leq d, \quad \sum_{j=1 \in d }P_{i,j} $$ and $i=1,....,d$. Let $ x_{0}$ be ...
user avatar
1 vote
1 answer
44 views

Comparison of Wolfe conditions to other "weaker" conditions or facts about optimization techniques for conceptual understanding

So in the book, it states that the first wolfe condition is the following, $\begin{equation}p^Tg_k\leq-\eta_0|||p|||g_k||\end{equation}$, where $g_k=\nabla F(x_k)$. Here it states that this is a ...
user avatar
0 votes
0 answers
9 views

Closed form solution of linear least square with penalty term and inequality constraint

I want to find a closed form solution of the following problem $$ \min_B \|V^T B- E \|^2 + \lambda \| B\|^2$$ $$s.t. V + B \geq 0, $$ where $B \in \mathbb{R}^{k \times n }$, $E \in \mathbb{S}^n$, and $...
user avatar
  • 710
1 vote
0 answers
24 views

Approximate first-order loss function with piecewise-linear functions in python

I'm trying to approximate the first-order loss function $$\mathbb{E}[max(d-y,0)]=\sigma \cdot (\phi(\frac{d-\mu}{\sigma})- \frac{d-\mu}{\sigma} \cdot (1- \Phi (\frac{d-\mu}{\sigma})),$$ where $d\in \...
user avatar
0 votes
1 answer
38 views

How can we solve this geometry problem using Lagrange Multipliers? It is must to use the given formula of area.

The problem is belongs to Mathematical Modeling with Excel book.
user avatar
0 votes
0 answers
15 views

Uniqueness of nonlinear least squares problem

Consider the problem $$\sum_{i=1}^m g_i^2(x) \to \min$$ with positive $\partial g_i/\partial x_k$, $x \in \mathbb R^n$, $m>n$. Can this problem have non-unique local minimum? What if we solve this ...
user avatar
0 votes
0 answers
24 views

Gradient descent on a nonconvex nonsmooth function

Suppose we have a nonconvex function $f: \mathbb{R}^c \to \mathbb{R}$ that could potentially be nondifferentiable at some points (but still is continuous - e.g, sum of absolute value functions with ...
user avatar
1 vote
0 answers
12 views

Pontryagin Maximum Principle with terminal and initial conditions

Consider a control problem with Lagragian $L(t,x,u)$ (where $u$ is the control, $x \in \mathbb{R}^d$ the state) and dynamics $\dot{x}=f(x,u,t)$. I have mostly seen problems in which the dynamical ...
user avatar
  • 41
1 vote
1 answer
79 views

Norm $\|x\|$ defined as the sum of largest $r$ absolute values, how to write $\min \|A x-b\|_{2}^{2}+\|x\|$ as QP?

This is a question from Boyd Convex Optimization, Additional Exercise 5.31 In this problem, $r$ is an integer between 1 and $n$, and $\|x\|$ denotes the norm $$ \|x\|=\max _{1 \leq i_{1}<\cdots<...
user avatar
  • 23
0 votes
1 answer
14 views

Showing that a maximum exist for a semi lower sequentially continuous mapping from Hilbert space to R

Let H be a Hilbert space over $R$ , $r > 0$ and $F ∈ C^1(H, R)$ such that: 1)−F is weakly sequentially lower semicontinuous 2) $DF(u) = 0$ implies $u = 0$ (this is the Frechet derivative) 3) $F(0) =...
user avatar
  • 97
0 votes
0 answers
15 views

Convex optimization with linear constraints. Can I solve it through KKT?

I have a constrained convex optimization problem with linear equality and inequality constraints. Minimize \begin{equation} \label{eq:costf} f(x_1,\dots,x_m) = \sum_{i=1}^m \frac{1}{x_i} \end{...
user avatar
1 vote
1 answer
76 views

Algorithms/Solvers for Hard Constrained Non-Linear Optimization Problems - Model Predictive Control Example

I have an autonomous robotic swarm path planning/control problem where a set of "leader" robots have predefined (nontrivial) dynamics in the control set, and "follower" robots are ...
user avatar
2 votes
0 answers
11 views

A question on the projection step of Generic Adaptive Method: $x_{t+1} = \Pi_{\mathcal{F},\sqrt{V_t}} (\hat{x}_{t+1}).$

I am reading the paper "ON THE CONVERGENCE OF ADAM AND BEYOND". In this paper, they proposed the following framework of adaptive methods. I was confused on the last step: $x_{t+1} = \Pi_{\...
user avatar
  • 601
1 vote
1 answer
25 views

Trouble about first order optimality conditions for programming problems with equality constraint.

I am having trouble understanding the following question. Question. Take the following non-linear programming problem. \begin{equation*} \min f(x_1,x_2,x_3) = x_1^2-3x_1x_2+x_2^2+x_3^2 \\[.15cm] \text{...
user avatar
  • 599
0 votes
1 answer
31 views

Maximum of $\textbf a^{ T} \textbf x$ over $ B[0,1]$ is $\textbf x^*=\frac{\textbf a}{\lVert \textbf a \rVert}$

Let $\textbf a \in \mathbb R^n$ be a nonzero vector.Show that maximum of $\textbf a^{ T} \textbf x$ over $ B[0,1]=\{\textbf x \in \mathbb R^n: \lVert \textbf x \rVert \le 1 \}$ is attained at $\...
user avatar
1 vote
0 answers
36 views

How do you represent the number of iterations in the formulation of an optimization problem?

Let's say I want to minimize some function for f(x), with respect to x, in the minimum number of iterations. How would I represent the number of iterations in the formulation of this optimization ...
user avatar
  • 133
0 votes
1 answer
22 views

Excel Solver Linear Optimization : Formula Debugging

I am trying to get a optimization model to work correctly. The background is to use the solver to find a circuit (AC or DC) that would minimize cost. I am trying to use binary variables so the ...
user avatar
  • 1
0 votes
1 answer
37 views

How to argue that $\min_{u\in \mathbb{R}^n} \frac{1}{2}\|u-u_0\|^2+\alpha S_\mu(u)$ has a unique solution

I am stuck at the following exercise: Consider a signal $u^*$ and a noisy signal $u_0$. I need to argue that the following problem $$\min_{u\in \mathbb{R}^n} \frac{1}{2}\|u-u_0\|^2+\alpha S_\mu(u)$$ ...
user avatar
  • 2,462
0 votes
0 answers
16 views

Can Constraint Qualifications guarantee non-negativity of Lagrange multipliers for any first-order stationary points?

Let the problem $$\begin{cases} \min f(x) \\ \text{s.t.}\\ g_i(x) \leq 0 \end{cases}$$ and its Lagrangian: $\mathcal{L}(x, \mu) = f(x) + \sum_{i}\mu_i g_i(x)$. Consider a stationary point of the ...
user avatar
  • 115
0 votes
0 answers
15 views

Does Newton method with line search accept unitary step close to the solution?

So Newton's method has local convergence, which means that close to the solution the unitary steps should suffice for convergence. On the other hand, Newton's method can attain global convergence ...
user avatar
  • 929
0 votes
1 answer
23 views

Exact VS Inexact Line search

I can't catch what is the difference between Exact and Inexact Line search algorithms. Could you help me? Thanks!
user avatar
0 votes
0 answers
23 views

Need help in using optimization theory to solve variables

$$\lvert H(f)\rvert=K\lvert\frac{1}{t_1^4}e^{-j2\pi ft_1}+\frac{1}{t_2^4}e^{-j2\pi ft_2}+\frac{1}{t_3^4}e^{-j2\pi ft_3}\rvert (1)$$ where K is constant,$|.|$means amplitude.I have the image of the ...
user avatar
0 votes
1 answer
21 views

The Proximal (Prox) Operator of the $ {L}_{0} $ Pseudo Norm Function

What is the Proximal Operator ($ \operatorname{Prox} $) of the Pseudo $ {L}_{0} $ Norm? Namely: $$ \operatorname{Prox}_{\lambda {\left\| \cdot \right\|}_{0} } \left( \boldsymbol{y} \right) = \arg \...
user avatar
  • 7,449
1 vote
0 answers
22 views

Problem with second order optimality in KKT

I'm solving an optimization problem with KKT method and I've found set of solutions for the problem that meat KKT conditions but now I'm trying to use second order optimality condition for my answer ...
user avatar
0 votes
0 answers
52 views

Can we substitute this KKT condition into this optimization problem to reformulate the optimization problem?

Suppose I have the following optimization problem: \begin{equation} \min\limits_{\mathbf{x},\mathbf{y}} f(\mathbf{x},\mathbf{y}) \qquad \qquad \qquad (1) \end{equation} It is already known that the ...
user avatar
  • 161
0 votes
0 answers
14 views

Optimization Algorithm for Continuous Objective but Binary Nonlinear Constraints

Is there an derivative-free local optimization algorithm for a continuous function with nonlinear constraints, where the constraints are binary? In other words: $$ \max_{x \in \mathbb R^n} f(x) $$ ...
user avatar
  • 1
1 vote
1 answer
41 views

Paradox definition of a function and its domain

I am currently reading and working through a chapter of "Convex analysis and nonlinear optimizations" by Borwein and Lewis (Chapter 3). At the beginning of the chapter I came across this ...
user avatar
1 vote
1 answer
56 views

Is the $\operatorname{argmin}$ of a uniform strongly convex function continuous?

Let $f:\mathbb{R}^n \times \mathbb{R}^m \rightarrow \mathbb{R}$ be a continuous function, and assume that $f(x,y)$ is $\mu$-strongly convex in $x$, for some $\mu>0$ and for any $y \in \mathbb{R}^m$...
user avatar
0 votes
0 answers
31 views

Solving a nonlinear optimization problem

I'm trying to solve the following problem: Let $\varphi_i: \mathbb{R}^n \to \mathbb{R}$ be continuously differentiable and $\varphi_i( {\bf 0} ) = 0$ for all $i=1,\dots,r$. Assume that $x^*$ is a ...
user avatar
1 vote
1 answer
39 views

One-dimensional heat conduction

I was reading a paper about one-dimensional heat conduction problem and I get stuck in one expression that I couldn't understand how to calculate. First, they define the heat conduction problem as ...
user avatar
  • 23
0 votes
0 answers
15 views

Transforming an unconstrained optimization problem into a two-step constrained optimization problems

I have the following problem: Problem 1: $\arg\min_{x,\vec{y}} g\left(\frac{f(x, \vec{y})}{\|\vec{y}\|}\right)\|\vec{y}\|$ where $g$ is monotonically increasing, $f$ is convex and $\|\cdot\|$ is some ...
user avatar
  • 1
4 votes
0 answers
31 views

Non-trivial example of a local optimum without satisfying Karush-Kuhn-Tucker (KKT) optimality conditions

I am looking for an example of a locally optimal point of the nonlinear program: $$\{ \min f(x) , g_i(x) \geq 0 \}$$ that does not have a singleton feasibility set. e.g. not the example of $\{ \min x, ...
user avatar
  • 115
0 votes
0 answers
58 views

How to determine the optimal solution for a parametric LP given data

I am trying to solve the problem in the image attached below. To do this, I first determined that the student with the correct values is Student C. I did this as follows: I let $x_1^*$ be an optimal ...
user avatar
0 votes
1 answer
28 views

What is the meaning of "local optimality" in NLP mentioned in Constraint Qualification?

Most statements of Constraint Qualification I have found in the literature mention a locally "locally optimal solution" of the problem: $$ \begin{cases} \min f(x) \\ \text{s.t.}\\ g_i(x)\leq ...
user avatar
  • 115
0 votes
1 answer
30 views

Generalizing regular polyhedra by repelling points on a sphere

Find the arrangement of $N$ identical point charges on a sphere. For uniqueness, assume one charge sits on the north pole and another one lies on a fixed latitude of the sphere Given a circumference ...
user avatar
  • 553
0 votes
0 answers
21 views

converting If conditions to linear constraints

I have an optimization problem and I want to convert the following if conditions to linear constraints: If $(y_1 > U_1)$ and $(m_1)$ and $(E_1)$ then $x_1=1$ If $y_2 > U_2$ and $(m_2)$ and $(E_2)...
user avatar
  • 1
0 votes
1 answer
35 views

Is there an "easy" way to find the minimum of $\frac{f(x) + g(y)}{x+y}$ where $f$ and $g$ are convex but $\frac{f(x) + g(y)}{x+y}$ is not?

Suppose I have a problem of the form $$min_{x \geq \epsilon, y \geq 0} \frac{f(x) + g(y)}{x+y}$$ subject to some (convex) inequality constraints and some affine equality constraints, and where $f$ and ...
user avatar
  • 1,492
2 votes
0 answers
28 views

$(w,h)=\arg\min_{x,y}\|A-xy\|^2_F,$ where $x,y$ are nonnegative vectors

Suppose $A\in \mathbb{R}^{m\times n},x\in\mathbb{R}_+^{m\times 1},y\in\mathbb{R}_+^{1\times n}$. $A$ has exactly one negative entry and others are nonnegative. Consider the problem $(w,h)=\...
user avatar
  • 829
1 vote
1 answer
34 views

How can I convert a non-linear constraint to a linear constraint for the mixed integer programming?

I have a nonlinear constraint: $\sum\limits_{i\in N}\sum\limits_{j\in J} A_{ijt}\times Z_{ijt}\geq \sum\limits_{i\in N}\sum\limits_{j\in J} D_{ij} \hspace{0.5cm} \forall{t}$ Here, $Z_{ijt}$={0,1}; $...
user avatar
  • 15
0 votes
0 answers
16 views

How can I design kernels for the Fourier Transform which minimize L1 distance for the truncated transform of particular functions?

Background: The $L_1$ norm has gotten much attention the last 15 years or so. Especially with the advances in numerical methods for optimizing with regards to it. Total Variation, Lasso and other ...
user avatar
  • 24.1k
0 votes
0 answers
5 views

Optimization Algorithm for efficient stochastic evaluation of blocks

Are there stochastic optimization algorithms that benefit from evaluating the target function at multiple, predefined values at the same time? Say I have a one-dimensional, smooth, and bounded ...
user avatar
  • 123
0 votes
0 answers
19 views

What would be a good loss function for this problem?

The problem is as follows, you have an implicit function $f:\mathbb{R}^2 \rightarrow \mathbb{R}$. You have a square region that overlaps with the boundary of the surface described by the iso line $f(x,...
user avatar
  • 2,269
2 votes
0 answers
26 views

How come nonlinear optimization problems need careful choices of initial parameters but neural networks appear to not have this issue?

When I run some nonlinear optimization code - I often encounter people saying that there is no global nonlinear optimization code that is guaranteed to reach a global maxima. Instead it is recommended ...
user avatar
0 votes
0 answers
30 views

How to understand the KL-divergence between two non-negative matrices

I am reading Non-negative Matrix algorithm using KL-divergence as metric. The KL-divergence is known as $D(P,Q)=\sum_i P(i)log\frac{P(i)}{Q(i)}$ for discrete distribution. However, the KL-divergence ...
user avatar
0 votes
0 answers
13 views

linear fit to 2D wrapped phase

Suppose we have a number of noisy space-phase data, i.e., $(x_i,y_i,\phi_i)$, and we know they are subject to a 2D linear phase pattern. That is to say, these data can be fitted using a 3-parameter ...
user avatar

1
2 3 4 5
55