Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [nonlinear-optimization]

A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for trying to solve particular problems.

1
vote
0answers
16 views

Hicksian Demand Function

Derive Hicksian demand for - $$u(x,y) = ax+ b\ln(y)$$ Explain in words what they mean? I solved the problem with the Lagrange Multiplier Method and found Hicksian demand for $x$ only. Solution: ...
0
votes
0answers
12 views

Conditions to apply Lagrange's Multiplier method

Can we always apply the Lagrange Multiplier method for constrained optimization? If not, what are the conditions to apply Lagrange Multiplier? What is the alternative method for constrained ...
0
votes
0answers
8 views

duality of a program on space of measures

let $\mathbb{S}^{d-1}$ be the unit sphere on $\mathbb{R}^d$ and $\mathbb{P}(\mathbb{S}^{d-1})$ be the space of probability measures on $\mathbb{S}^{d-1}$. Given a probability measure $\pi$ on the ...
0
votes
2answers
37 views

What is the role of Tikhonov regularization in optimization?

Suppose I have the following objective function $$ L = \frac{1}{|X|}\sum_{x \in X} \| \hat{y} - y \|^2_2 + \lambda \|w\|_2^2 $$ where $X$ are my data, $\hat{y}$ the prediction, $y$ the target, $\...
0
votes
0answers
6 views

How to translate estimated model parameters when fitting centered and scaled data?

I routinely use a non-linear curve fitting tool to fit data according to a user prescribed model / function. One piece of advice that I commonly see around non-linear curve fitting is about data ...
0
votes
1answer
16 views

Alternative to minimum squared method

I'm writing a Matlab code where I need to find the minimum of a function and to do that I compute the distance between two points $$ d = \sqrt{\sum (x_{ideal} - x_{real})^2 + (y_{ideal} - y_{real})^2}...
0
votes
0answers
28 views

How to linearize a $1$-norm equality constraint?

Let $x, y \in \mathbb{R}^n$ be fixed vectors of $1$-norm $C$. My optimization problem is the following $$ \underset{\beta \in \mathrm{R}^n}{\text{minimize}} |\beta |_M \\ \text{subject to} \\ |x+...
0
votes
0answers
24 views

Optimization of a function of time-series data.

Consider a non-linear function $f(x_t; \theta)$. I have datapoints $\{x_t, y_t\}$ and I am trying to optimize $\theta$ for some cost function $C(y^\prime_t,y_t)$ where $y^\prime_t = f(x_t;\theta)$. I ...
0
votes
1answer
18 views

The boundness of a polyhedral set implies that $y \in \mathbb{R}^n $ is equal to zero

I'm solving a problem of nonlinear programming. The problem says: Let $S_1=\{x:A_1 x\le b_1\}$ and $S_2=\{x:A_2 x\le b_2\}$ be nonempty. Define $S=S_1\cup S_2$ and $S'=\{x: x=y+z, A_1y\le b_1\...
1
vote
0answers
29 views

How to prove that the subgradient of a dual function contains the equality constraint for closed and convex function?

Apologies for the fundamental question. But I am am just trying to understand the pieces. Let us consider a minimization problem \begin{align} \text{minimize}_{x \in \mathbb{R}^n} \quad & f(x) \\ ...
0
votes
1answer
16 views

What is a nontrivial minimizer?

I came across a statement that x is a nontrivial minimizer of some function, but couldn't find a definition of "nontrivial minimizer" on the Internet. Can anyone help point out some references for ...
1
vote
0answers
17 views

Efficient algorithms for minimization of bi-linear matrix expressions?

Say I want to solve $$\min_{\bf x}\|\bf Ax-b\|_2^2$$ This problem is well defined and easy to solve using linear least squares. But what if we suddenly have $$\min_{\bf x,A}\|\bf Ax-b\|_2^2$$ ...
0
votes
0answers
19 views

Maximum likelihood estimator of the parameter of randomness in Watts and Strogatz's model (1998).

According to the paper Menezes, M. B., Kim, S., & Huang, R. (2017). Constructing a Watts-Strogatz network from a small-world network with symmetric degree distribution. PloS one, 12(6), e0179120,...
0
votes
0answers
44 views

Optimization Methods in Banach Spaces

does anyone know if there's a theory for the following problem: Optimize the task $\begin{align*} T_\phi(\tilde{u})&=\inf\limits_u T_\phi(u)\\ Au&=b\\ u&\in L^p(\Omega),\,\Omega\subset \...
0
votes
1answer
12 views

Gauss Newton Direction

I am working on Gauss Newton direction but I have a hard time understanding: $\ J^tJ p = - J^tr $ What I searched and understood is as follow: $\ - J^tr $ is in face $\ - \nabla f $ which is like ...
0
votes
0answers
34 views

Reference request. Rigorous numerical optimization

I am looking for texts on Numerical optimization that are closer to Analysis definition-theorem-proof style texts. EDIT: This is my first acquaintance with numerical optimization. My institution ...
2
votes
0answers
47 views

How to solve a large scale convex optimization problem?

Consider the following convex optimization problem in vectors $x$ and $y$ $$\begin{array}{ll} \text{minimize} & f(x,y)\\ \text{subject to} & g_1 (x)\le 0\\ & g_2 (y)\le 0\end{array}$$ ...
0
votes
0answers
13 views

What if provide a good initial point for penalty convex-concave procedure?

Recently, I am working on penalty convex-concave procedure (PCCP) to solve some optimization problems. I learned from S. Boyd's great book "Variations and extension of the convex–concave procedure" ...
1
vote
0answers
29 views

Show that global minima exists or not

My function is defined $f(x,y) = 2x^3 + 3y^2+3x^2y-24y$. I found the critical points $(x,y) = (0,4), (-2,2)$ and $(4,-4)$. I also showed that $(0,4)$ is the local minimum by showing that the Hessian ...
-4
votes
0answers
25 views

I look for C/C++ code for solving LMA algorithm in fsove function in MatLab for this below problem [closed]

I look for C/C++ code for solving $$ \left\{ \begin{array}{ccc} A\cdot\operatorname{cosd}(x)+B\cdot\operatorname{sind}(x)-B\cdot\operatorname{cosd}(y)-C\cdot\operatorname{sind}(y)&=&0 \\ D\...
0
votes
2answers
31 views

Constrained minimization of multi-variable trigonometric function

Let f(x,y) = $\sin^2(x)$+$\arctan^3(xy)$. For every positive integer $n$ and every $M \ge 0$, find $$ \min_{(x,y)\in S_{n,M}} f(x,y), $$ where $S_{n,M} =[0,n \pi]×[−M,M]$,and determine where this ...
0
votes
0answers
37 views

Is this objective on a optimization problem convex?

Is this objective convex? If not is there a way to make it convex? I am trying to optimize the values of the matrix A for some defined values of the vectors x,y,w $ min_A: \sum_{m} \Bigg({\dfrac{x_m ...
0
votes
0answers
15 views

Is there a way to show that a coercive and convex function is also strongly convex? [closed]

Is it true and is there a way to prove it if true? One counter example would be helpful if false
0
votes
0answers
8 views

What is the purpose of the contrast function in independent component analysis, specifically fast ICA?

What exactly is meant by a contrast function in this context? How does it differ from the objective function? Why is it that several contrast functions can work here?
2
votes
0answers
69 views

Is $f(x,y)=\left(\frac{x^a(1-y)}{(1-x)}-\frac{y^{a}(1-x)}{(1-y)}\right)\frac{1}{x-y}$ convex?

Is it possible to prove that, $$f(x,y)=\left(\frac{x^a(1-y)}{(1-x)}-\frac{y^{a}(1-x)}{(1-y)}\right)\frac{1}{x-y}$$ is convex in the following range: $0<y<x<1$, where $a\ge2$ is an integer ...
0
votes
0answers
30 views

When a constrained optimization problem and its Lagrangian are equivalent?

Assume both $f(x)$ and $g(x)$ are non-negative and non-convex scalar functions. Further assume that for any $x_1,\, x_2$ it holds $f(x_1) \le f(x_2)$ if and only if $g(x_1) \ge g(x_2)$. Can we say ...
2
votes
0answers
36 views

Find optimal weight vector $\sum_i \log \left(1 + \exp\left\{ -t_i \left(w^T x_i\right)\right\} \right) + \mu \|w \|_2^2$?

Let the function is, \begin{align} f(w) &= \sum_i \log \left(1 + \exp\left\{ -t_i \left(w^T x_i\right)\right\} \right) + \mu \|w \|_2^2 \ , \end{align} where $t_i \in \mathbb{R}$, $w, x_i \in \...
1
vote
3answers
53 views

Slow convergence of gradient descent for a strictly convex quadratic

Let $0 < \lambda_1 \leq \lambda_2 \leq \ldots \leq \lambda_n$ and let $f: \mathbb{R}^n \to \mathbb{R}$ define by $$ f(x) = \frac{1}{2}x^TMx $$ where M is \begin{bmatrix} \lambda_1 & 0 &...
0
votes
0answers
29 views

Non-convex QCQP with embedded variable

I have the following problem whose optimal solution (if possible), I would like to find. $\min_{\mathbf{f}} \left\| \mathbf{L}_1 \mathbf{f} \right\|^2_2 + \left\| \mathbf{L}_2 \mathbf{f} \right\|^2_2 ...
1
vote
1answer
64 views

Non linear optimisation with min functions

I have the following nonlinear optimisation problem under bounds constraints and involving $\min$ functions and the euclidean norm in the objective function : $$\underset{a,b,c,d}{\min} \Big\Vert \...
1
vote
1answer
15 views

Reference on Lipschitz property of the infimum of a family of Lipschitz functions

I can prove the following fact: the infimum, or supremum, of any family of L-Lipschitz functions is L-Lipschitz, as long as the constant L is fixed. However, since this is a very basic result, I am ...
0
votes
2answers
62 views

Projectile Motion, finding the optimal angle

I've been studying multivariable calculus the last 2 weeks, and I understand (I think) how to optimize 2 variable equations through normal optimization and constrained optimization via Lagrange. I ...
0
votes
1answer
56 views

Optimization model where a certain conditions affect objective rather than being a constraint

I have a minimization problem related to packing. I have to distribute a given amount of items from different sets to different packs. Each pack must be either empty or have a certain minimum of items....
4
votes
0answers
51 views

From infinite dimensional function space to n-dimensional real space

I am an engineering student who works in the field of optimal control. Problems in this field are typically framed in infinite dimensional Sobolev space, $W^{k,p}(\Omega)$, as: \begin{equation} \...
1
vote
2answers
52 views

Is modified Newton's Raphson method redundant?

I have been recently taught Newton's method for finding roots of non-linear equations. I was told in class that if the multiplicity of the root is more than 1, then the order of convergence is not ...
3
votes
1answer
58 views

Solve $\min_{x} \frac{1}{N}\sum_{i=1}^{N} f_i(x_i) + g(x)$ $\ {\rm s.t.} Ax = b$; $x = [x_1,\ldots,x_N]^T$ and $A \in \mathbb{R}^{M \times N}$.

An optimization problem: \begin{equation*} \begin{aligned} & \underset{ x \in \mathbb{R}^N }{\text{minimize}} & & \frac{1}{N} \sum_{i=1}^{N} f_i\left(x_i\right) + g(x) \\ & \text{...
1
vote
1answer
22 views

Cross entropy loss is not twice differentiable?

Pardon if this seems off topic, I was reading this recent theory paper in machine learning by Kenji Kawaguchi and Leslie Pack Kaelbling https://arxiv.org/pdf/1901.00279.pdf and the authors seem to ...
0
votes
1answer
21 views

Methods for solving nonlinear constraints quadratic programming

Are there any other methods to solve nonlinear constraints quadratic programming? I have known that some effective numerical methods, i.e, SQP and Gauss pseudospectral method and some heuristic ...
1
vote
0answers
40 views

Nonlinear optimization with parametric constraint

Is there any way of reformulating the following problem so that it can be solved by means of e.g. Matlab's fmincon? $\min f(x_1,\dots,x_n)$ subject to $c(x_1,\dots,...
0
votes
1answer
25 views

Gauss-Newton local convergence

Is it possible for the Gauss-Newton algorithm to converge to a local minima? How does convergence of Gauss-Newton to the correct minima compare with gradient decent and Levenberg-Marquardt?
4
votes
0answers
16 views

Optimization of nonlinear $f(x)$ where $x$ is a vector of binary variables

I'd like to find a solution (potentially approximate) to the problem $$ \max_{x_{i,j}} \sum_{k=0}^K\left[ 1 - \prod_{i=1}^{I} \left(1 - \prod_{j=1}^{J}(1-b_{k,i,j} \, x_{i,j}) \right)\right] $$ ...
0
votes
1answer
30 views

Optimizing a sum of functions

I'm not an expert in optimization, but I am currently working on a problem where I need to maximize/minimize a function of the form, \begin{equation*} g(\alpha_0, \alpha_1) = \displaystyle \sum_{i=1}^...
2
votes
0answers
26 views

Efficient SQP with more equality constraints than parameters

My question is both a math and (computer) programming one so answers related to either are fine. Problem Setup I have the nonlinear programming problem $$ \begin{aligned} \min \;\; &f(X) \\ \...
1
vote
0answers
39 views

Binary Optimization Problems that can be easily solved?

As far as I have researched, even linear programs with binary constraints on the decision variables are in general NP hard. However, I wounder if there are some (non-trivial) binary optimization ...
1
vote
0answers
40 views

Rewrite $ \min_{q\in Q_0} \sum_{x=1}^X |m_x(q)| $ with linear objective function

I have the following optimization problem $$ \min_{q\in Q_0} \sum_{x=1}^X |m_x(q_{1})| $$ where $q\equiv (q_1,q_2)$ is a vector that should satisfy a bunch of non-linear constraints collected in $...
0
votes
0answers
20 views

Prove Unique Lagrange Multipliers Equality Constraint

I am working through some old test papers in preparation for exams an am trying to scout out potential sneaky questions that might be asked. I've stumbled across this one. Would you please verify or ...
0
votes
0answers
16 views

Transform the problem to EQ constrained problem with simple bounds

\begin{align} min && x_1^2 + x_2^2\\ s.t. && (x_1 -3)^2+1 \leq x_2\\ &&x_1-2x_2+2=0\\ &&x_2 \geq0.5 \end{align} SOLUTION. \begin{align} min && x_1^2 + x_2^2\\ ...
4
votes
1answer
64 views

Orthogonal Projection onto the Weighted $ {L}_{2} $ Norm Ball

Projection onto the $\ell$2 norm ball is known. Let the norm ball set reads $C = \left\{x \in \mathbb{R}^n: \left\| x - c \right\|_2^2 \leq d^2 \right\}$, where $c \in \mathbb{R}^n$ and $d \in \...
0
votes
0answers
35 views

Motive of Conjugate Gradient method.

It is known that the solution to the linear system $Ax=b$ where $A$ is symmetric and positive definite is the minimizer of the quadratic function $f(x)= \frac{1}{2} x^T A x - x^T b$ We can solve it ...
0
votes
0answers
16 views

Concerning the idea of Trust Region methods

As far as I understood is that the idea of TR methods is that at the current iterate $x_k$ we build a model "usually quadratic", of the objective function $f$ to be optimized, $m_k(s)$ of $f(x_k +s)$ ...