A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

1
vote
1answer
105 views

Proving Convergence of a Derivative Free Method

Hopefully this question is on topic for the mathematics community (rather than the statistics) since the optimization method I am using relies on a statistical model. Well I am building an algorithm ...
0
votes
1answer
25 views

Nonlinear optimization related to symmetric functions

Suppose $f(x,y)\geq 0$ is integrable and symmetric in $x$ and $y$, i.e.$f(x,y)=f(y,x)$. Consider the following nonlinear optimization problem $$\max F(a,b)=\int_0^a\int_0^bf(x,y)dxdy,$$ Subject to $ab=...
0
votes
1answer
25 views

Optimization involving integrals with varying limits

What are the common methods and tools to tackle optimization problemsinvolving integrals. To be precise lets consider the following optimization problem that I came across with: $$\text{maximize}\,\,F(...
0
votes
1answer
29 views

Positive definite and semi definite in non linear programming [duplicate]

How can I prove the following. Suppose that A is a square matrix and suppose that there is another matrix B such that $A=B^TB$. a)Show that A is positive semi definite b)Show that if B has ...
1
vote
1answer
41 views

Karush-Kuhn-Tucker NLP

Consider the nonlinear program Minimize: \begin{align}f(x,y) = \frac{1}{2}x^2 - 10xy + 10y^2\end{align} Subject to: \begin{align}2x +y^2 &\le 5 \implies g_1(x,y)=2x + y^2 -5 \le0 \\ x^2 -...
6
votes
1answer
184 views

How is the Lagrangian related to the perturbation function?

Given a convex programming problem $$\begin{align*} \text{minimize} &\quad f(x) &\\ \text{such that} &\quad g_i(x) \leq 0 & i=1\dots k\\ & \quad h_j(x) = 0 & j= k+1\dots n \...
0
votes
2answers
38 views

How to make a non-linear problem linear?

I have the following constraint which is the product of multiple binary variables: $1- \prod_i^n (1-(c_i x_i)) >= T$ where $x_i$ is a binary variable, $c_i$ is a constant and $T$ is a constant too....
2
votes
0answers
36 views

solving a collaborative filtering problem

I was reading this paper Bell RM, Koren Y. Scalable collaborative filtering with jointly derived neighborhood interpolation weights. Proc - IEEE Int Conf Data Mining, ICDM. 2007:43-52. doi:10.1109/...
0
votes
2answers
94 views

Nonlinear LS regression

• Problem formulation I have to fit the following nonlinear model to a dataset: $$f(x)=\frac{C_1 \cdot a}{a^2 + C_2 \cdot x^2}$$ $a$: fitting parameter $C_1, C_2$: Given constants I can't apply ...
0
votes
0answers
85 views

references: L-BFGS rate of convergence

I was trying to find results about the rate of convergence for the L-BFGS algorithm (in the nonlinear case). What I end up with so far is that the BFGS-Algorithm converges Q-superlinearly this 50 ...
2
votes
0answers
23 views

Linear space transform transformation based on covariance?

I have a linear space of n dimensions with non-overlapping groups characterized by different variation (different covariance matrices). Is there a way to deform non-linearly the space according to an ...
1
vote
0answers
23 views

quasi-newton method converges in at most n+1 iterations

Given $B_{k+1}$ be obtained from $B_k$ using the symmetric rank-one update formula. Assume that the associated quasi-Newton method is applied to an n-dimensional, strictly convex, quadratic function, ...
1
vote
0answers
53 views

Nonlinear Least Squares vs. Extended Kalman Filter

What is the relationship between nonlinear least squares and the Extended Kalman Filter (EKF)? I've learned both topics separately and thought I understood them, but am now in a class where the EKF (...
0
votes
0answers
29 views

Is it true that there exist exactly ${k\choose n}$ bases that lead to this basic feasible solution?

Let a matrix $A=\left(A_{ij} \right)_{k\times n}$, with $A_{ij}\in\mathscr{R}$, and $$\mathrm{P}=\{ \mathtt{X}\in \mathscr{R}^n \,|\, A\mathtt{X}\ge b\}. $$ Suppose that at a particular basic ...
2
votes
1answer
68 views

KKT condition - minimization problem

$y^2-8 \ln(x+4)\rightarrow$ min, such that $-x^2 -y^2+9 \geq 0, y \geq 0$ *I have to find all possible optimal points.* Lagragian function is: $L(x,y,γ_1,γ_2) = y^2 - 8\ln(x+4)+γ_1(x^2+y^2-...
0
votes
1answer
25 views

Linearizing a constraint for ILP

I have binary variables $x_{ij}$. One of my constraint is $$\sum\limits_{i}\sum\limits_{j} x_{ij}*f_i(\sum\limits_{j}x_{ij})\leq B \ $$ where my $f_i()$ is implemented as a table. Will it be ...
1
vote
0answers
14 views

Problem with nonlinear equation system

I need to calculate two coefficients: k and n. This involves solving two equations for k and n: $$\frac{(8 \pi d)}{\lambda }{kn}=\frac{\text{$\triangle $I}}{I}$$ $$\text{$\triangle $l}_2=\left\{\...
0
votes
0answers
25 views

How to find Common (invariant) Subspace between more than two Hankel Matrices?

Note: I am not a mathematician but a control engineer. A general nonlinear $n_{a}^{th}$ order discrete-time state-space model is described by the following equations: \begin{align} ...
1
vote
0answers
29 views

Choosing a non-convex global optimization algorithm based on the number of permitted steps

Can anyone comment on the most suitable approach for the following optimization problem: We are given finite bounds for a set of $n$ real-valued parameters of an unknown deterministic function. The ...
0
votes
0answers
47 views

To solve a non-linear equation system with huge amout of variables analytically or numerically

I have such a nonlinear equation system $x_i=\frac{\sum_{j\neq i}a_{ij}\times\sum_{k\neq i}x_k}{\sum_{k\neq i}x_k-\sum_{j\neq i}a_{ij}}$ where $a_{ij}$s are known coefficients in $[0,1]$. And $x_i$s ...
0
votes
2answers
44 views

Why is an affine set convex?

I wanted to know why do we say that an affine set is convex? From what I understood, if we take two points $x_1$ and $x_2$ $\in \mathbb{R}$, then, the affine set $A$ defined by these two points will ...
0
votes
2answers
90 views

Lagrange's method question

Find the extreme values of the function $f(x,y,z)=x^2+y^2+z^2$ subject to the condition $xy+yz+zx=3a^2$. I tried to solved it by Lagrange method and got $3$ equations. \begin{align*} &2x+k(y+z)...
0
votes
1answer
24 views

Estimating errors from optimization? (Genetic algorithm or otherwise)

I have a vector of observations $\vec x_{\text{obs}}$ that have been measured with known uncertainties $\vec \sigma_{x}$. I have a model $f$ that takes parameters $\vec \theta$ and produces values $...
1
vote
0answers
37 views

nonlinear Lasso with constraints

Just encountered a very strange problem in my research. It's a nonlinear lasso with constraints. The optimization is $\min $ $\sum_{t=1}^{T}f\left( y_{t},x_{t};\beta \right) +\lambda \left\Vert \...
0
votes
1answer
30 views

Can an unfeasible solution be optimal in an LPP

In a linear programming problem, Is it possible to have an unfeasible solution that is optimal?
0
votes
0answers
38 views

Semidefinite programming formulation for a simple minimization

I am trying to formulate following problem (with some constraint) as a semidefinite programming problem (SDP), \begin{equation} \text{minimize } ~~ -a^T B^{-1} a \end{equation} where $B$ is a ...
0
votes
0answers
18 views

Theory of Adaptive Interaction

Does someone has knowledge on the theory of adaptive interaction? I have read that is a simple and effective way to perform gradient descent in the parameter space. I need to implement a adaptive PID ...
0
votes
1answer
72 views

KKT condition just binding and inactive

I have read the textbook saying that if both KKT and Lagrangian multiplier $\lambda$ are $0$, then the constraint is just binding, whereas if KKT multiplier is equal to 0, and Lagrangian multiplier is ...
0
votes
0answers
34 views

Constrained optimization of $||S - ABA^T||$

Given asymmetric matrix $S_{n\times n}$ we want to decompose it into $A_{n\times k}$ and $B_{k\times k}$ such that $S\approx ABA^T$. (Constraints: columns of B sum up to one while all elements are non-...
0
votes
1answer
69 views

Finding the value of coefficients of a equation using non-linear least square method.

I have the following data: x: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 y: 22 36 42 51 57 64 68 71 75 79 85 87 88 91 94 97 99 99 103 104 105 107 108 109 ...
3
votes
1answer
73 views

KKT conditions for a maximization problem

I have an optimization problem \begin{equation} \mathbf{w}^*= \text{argmax} \sum_{d=1}^{D}\log\left(\frac{|\mathbf{\hat{f}}_{d}^{H}\mathbf{w}|^{2}+A_d}{|\mathbf{\hat{f}}_{d}^{H}\mathbf{w}|^{2}+B_d}\...
1
vote
0answers
55 views

Division of linear functions in convex polytope

I am a computer scientist, and find myself needing the following lemma: If f(x)=(g(x))/(h(x)), where g and h are linear and positive with domain the convex polytope d, then extrema of f occur at ...
0
votes
0answers
44 views

Sufficiency of KKT problem

I have a continuous, twice differentiable function $f:\mathbb{R}^2\rightarrow \mathbb{R}$. My objective is to maximise the function $f$ subject to quasiconcave function $g_i(x,y)\le 0,\, i\in\{1,2\}\, ...
1
vote
0answers
31 views

Optimization when the function is not known, how generally it is performed?

I've a question on optimization (it is a general question). I have a problem $P$ where basically i have the analytic expression of the objective function, which is a multivariate polynomial (positive),...
3
votes
2answers
75 views

$g$ is coercive for $g(x)=x^TAx+b^Tx+c$

Suppose $A$ is a symmetric positive definite matrix $A\in \Bbb{R}^{n\times n}$, $b \in \Bbb{R}^n$, and c is a real number. Let $$g(x)=x^TAx + b^Tx + c$$ Show that $g$ is coercive. Because $A$ is ...
0
votes
1answer
72 views

Choose of the parameters related to the Wolfe-Powell conditions

Let $f\in C^1(\mathbb R^n)$ $x\in\mathbb R^n$ and $d\in\mathbb R^n$ with $$\langle\nabla f(x),d\rangle<0\tag{1}$$ Then, $t>0$ is said to satisfy the Wolfe-Powell conditions with parameters $\...
0
votes
0answers
34 views

mathematical optimization via lagrange multipliers (or other method)

Let \begin{eqnarray} g_k(x_1,\ldots, x_K) = x_k - \log \sum_{j=1}^K \exp x_j. \end{eqnarray} Notice that, for $i\neq k$ \begin{eqnarray} \frac{\partial g_k}{\partial x_i} = -\frac{\exp x_i}{\sum_{j=...
1
vote
0answers
19 views

Levenberg-Marquardt fitting with multiple $y$ for each $x$?

I am trying to implement LM for my research project. The functions is in the form of $f(x,params) = y$. The issue is that for each value of $x$, there are five values of $y$, even for same $params$. ...
1
vote
0answers
38 views

Computational methods to minimizing the norm of a matrix monomial.

Linear optimization solves the problem $$\min_{\bf x}\{\|{\bf Ax - b}\|_2^2\}$$ Edit: Some clarification Doing the derivation of the optimum, first expand the norm: $$\|{\bf Ax - b}\|_2^2 = ({\bf ...
2
votes
2answers
36 views

$f (x_1,x_2)=x_1^2+x_2^3 $ Finding if a local minimum exist

Why is $(0,0)$ not a local minimizer for $f (x_1,x_2)=x_1^2+x_2^3 $? Because $(0,0)$ is a critical point and Hessian matrix at $(0,0)$ is positive semi definite. Therefore isn't it a local minimum ...
1
vote
2answers
126 views

Finding global maximizers and minimizers

I want to find if global maximum or minimum exists in $ f(x,y)=e-^{(x^2+y^2)}$ I found that (0,0) is the only critical point. In the Hessian matrix $H_{(f)}(0,0)$ was negative definite and so (...
1
vote
2answers
176 views

How to determine Coercive functions

A continuous function $f(x)$ that is defined on $R^n$ is called coercive if $\lim\limits_{\Vert x \Vert \rightarrow \infty} f(x)=+ \infty$. I am finding it difficult to understand how the norm of ...
1
vote
0answers
10 views

Is s-energy, specifically logarithmic energy, based on function -log(d) or log(d) with distance d?

There was an earlier question about s-energy; I think I got the point. However, in various problem settings I see one is interested maximising pairwise distances between points and, there they say, ...
0
votes
0answers
23 views

Are the number of Employed bees and onlooker bees always the same in Artificial Bee Colony optimization?

I have been recently introduced to this Artificial Bee Colony algorithm. Lets consider the initial guess and assume that half of the bees are employed and the rest half are onlookers. Once the ...
4
votes
1answer
62 views

An optimization problem involving a probability density function

I have three time-series $\mathbf{x}_{1}, \mathbf{x}_{2}, \mathbf{x}_{3}$. I would like to find a linear combination of the time series, that is, some scalars $a_{1},a_{2},a_{3}$ such that the sum $$\...
0
votes
1answer
147 views

How can linearize the product of decision variables in ILP?

Here, we have something like this: R + (1-R)T + (1-R)(1-T)S + (1-R)(1-T)(1-S)Q = 1 where R, T, S, Q are binary decision variable How can I convert this ...
4
votes
1answer
60 views

Stuck on step in Lagrangian Problem

For $w,E$ column vectors, $i$ the vector of ones, and $\Sigma$ - an $n\times n$ positive definite symmetric matrix, I am trying to solve the following maximization problem: $$ \max_{\{ w\}} \left\{ \...
0
votes
1answer
20 views

Non-linear programming problem from a trapping region in a nonlinear ODE system?

I have the next function... wich describes a trapping region from a glycolysys model of Sel'kov... I need to minimize $z=h(u)=u^3(\frac{1}{A}-\frac{B}{A^2})+u^2(\frac{\epsilon B}{A^2}-1-\frac{\...
1
vote
1answer
24 views

Computationally inexpensive method to find a rotation which minimizes the norm of two tensor difference.

So I have two matrices ${\bf T}_1$ and ${\bf T}_2$, they are tensors in the sense that they can be built as $${\bf T} = \sum_{\forall i} a_i({\bf v_i}{\bf v_i}^T)$$ with positive real weights $a_i$ ...
0
votes
0answers
37 views

Global optimization methods where constraints are lipschitz functions

Is there any global optimization methods where objective function is nonlinear (not lipschitz) but constraints are lipschitz functions?