A non-linear optimization problem includes an objective function (to be minimized or maximized) and some number of equality and/or inequality constraints where the objective or some of the constraints are non-linear. Use this tag for questions related to the theory of solving such problems or for ...

learn more… | top users | synonyms

2
votes
1answer
523 views

Definition of tangent cone in continuous optimization .

Looking at the definition of tangent cone in continuous optimization : If $M$ is a open subset of $\mathbb R^n$ $x \in M$, The tangent cone of $M$ at $x$ is defined by $$\mathbb T (M, x) = \big\{d ...
1
vote
0answers
85 views

Some problems with a proof of the Farkas Lemma

The following is a proof of the Farkas Lemma that is creating me quite some problems. [I presented the all proof simply to point out the notation used by the author.] My problem is with the last part ...
1
vote
1answer
190 views

Minimizing a linear function on a strictly convex set.

All the theorems that I know considering the uniqueness of a solution to a minimization/maximization problem requires the strict convexity/strict concavity of the objective function. But consider the ...
0
votes
2answers
40 views

Maximum of convex functions

how can i proof that: If $f_1, . . . , f_m$ are convex functions,than function $F(x) = \max(f_1(x), \dots , f_m(x))$ is convex? thanx for help.
1
vote
1answer
197 views

Third and higher order optimality conditions?

In the derivation of first and second order optimality criteria for a vector $X^*$ to be a local optimum to an unconstrained problem, we ignore the higher order terms of Taylor's expansion as we ...
0
votes
0answers
83 views

How do I find out if a critical point of a function is a maximum or a minimum?

If I've found the critical point of a function defined in some constraint (perhaps using Lagrange multipliers and the like); how do I find out if it's a relative/global maximum/minimum of a function ...
0
votes
0answers
27 views

How to check if steepest gradient method will converge?

So I have this function $ f(x,y) = x^4 - 2x^2 +x + 4y^2 $ and I want to know if the steepest gradient method will converge if I pick an arbitrary point and apply said method. My initial thought ...
3
votes
3answers
62 views

Minimization of $log_{a}(bc)+log_{b}(ac)+log_{c}(ab)$?

I am trying to find the minimal value of the expression: $log_{a}(bc)+log_{b}(ac)+log_{c}(ab)$ I think experience gives that the variables should be equal, if so then the minimal value is 6, but ...
0
votes
0answers
56 views

Using Newton's method to find an optimized matrix

I'm trying to apply Newton's method to find a local optimum of the matrix $\Sigma$ to minimize the objective function: $f(\Sigma) = -\sum_{n=1}^{N}\left(-\ln{2\pi} - ...
0
votes
1answer
28 views

Help in Linear summation of n(any given number)

The original formula is this. We're computing the complexity of an insertion sort. How did the first formula turned into the second formula?
1
vote
1answer
55 views

Uniqueness of Solution to non linear polynomial equations given by lagranges method

When considering Lagrange's method of multipliers for finding maximal solutions to a set of non-linear equations, I have reached a set of 4 equations in 4 real unknowns, $(a,b,c,\lambda)$: ...
1
vote
0answers
37 views

Decomposition of a symmetric semi-definite matrix into sums of sparse symmetric semi-definite matrix

I'll first provide the background. Let $x\in\mathbb{R}^N$ be decomposed into $n$ non-overlapping blocks of variables $x^{(1)},\ldots,x^{(n)}$. We say that $f:\mathbb{R}^N\rightarrow\mathbb{R}$ is ...
1
vote
0answers
64 views

Fitting a sine using linear regression

If I have two functions $s_1 = A_1 \sin(\theta+\phi)$ and $s_2 = A_2 \cos(\theta+\phi)$ is it possible to fit a sine or a cosine using linear regression? I usually have much less that a period ...
4
votes
0answers
33 views

Nonlinear optimisation of Expectation

I am preparing for my exams and I can't get my head around the following question. I know there exists a general method for solving these problems but I don't know where to start. I would greatly ...
2
votes
2answers
64 views

$\sum\limits_{k=0}^{19} \sqrt{1+u_k^2} \rightarrow \min$

Solve $\sum\limits_{k=0}^{19} \sqrt{1+u_k^2} \rightarrow \min$, such that $x_0 = 0, x_{20} = 5$ and $x_{k+1} - x_k = u_k$. I think I know how to solve problems like these recursively, but I ...
4
votes
2answers
90 views

Maximize $\prod\limits_{i=1}^n m_i$

Someone visits a market where one hundred different types of fruit are sold. All types cost $1$ euro per pound. The utility the buyer receives from buying $m_1$ pounds of the first type of fruit he ...
0
votes
1answer
43 views

What is inexact steepest descent method

Is there anybody knowing what is the inexact steepest descent method for solving non-linear optimization problems? Any reference or formal definition available online? I was asked by someone, but ...
0
votes
1answer
281 views

KKT maximization problem

$x^2y \rightarrow$ max, such that $x^2 + 4xy \leq 1, x \geq 0$ and $y \geq 0$. I think I need to use the KKT conditions here. I did however not yet succeed in solving it, so could someone ...
2
votes
1answer
1k views

Taylor's theorem for vector valued functions

I'm reading about linear and nonlinear programming and on one page I have the following statment (I have highlighted the areas where I have problems and drawn questions for them in the bottom of it): ...
1
vote
2answers
212 views

What is a convex optimisation problem? Objective function convex, domain convex or codomain convex?

My teacher in the course Mat-2.3139 did not want to answer this question because it would take too much time. So what does a convex optimisation problem actually mean? Convex objective function? ...
1
vote
2answers
286 views

Why nonlinear programming problem (NLO) called “nonlinear”? What does “nonlinearity” actually mean? Is it “not linear” or something different?

My teacher in the course Mat-2.3139 presented the same definition as in Wikipedia for the nonlinear programming problem here but he did not specify what the nonlinearity actually means or what it ...
0
votes
1answer
141 views

(pseudo-/quasi-)convexitiy of ratio between quadratic and affine function

Let $X\subseteq\mathbb{R}^n$. I have the following function $f:X\rightarrow\mathbb{R}$: $$ f({\bf x})= \frac{c_1 + \sum_{i=1}^n a_ix_i +\sum_{i=1}^n b_ix_i^2}{c_2+\sum_{i=1}^n d_i x_i}\enspace.$$ All ...
1
vote
1answer
59 views

Help to understand the setting up of this Lagrangian

So..I understand up to step 4..but then there are these things I dont get, to start with , it says on (5) that the utility function depended only on the ratios p1/w p2/w ?? why does it say that? ...
1
vote
0answers
29 views

Nonlinear optimization using parallel input/output

I have a system that accepts a vector and returns a function value. The goal is to change the elements of the vector such that the function value is minimized using a derivative-free solver, eg. using ...
1
vote
0answers
55 views

Regression/compressive sensing with non-linear constrains where the coefficients are assumed to be integer or binary {0,1}

The following regression problem $$ \mathbf{y} = \mathbf{A}\mathbf{x} $$ where $\mathbf{y}$ is a $N\times 1$ column real vector, $\mathbf{A}$ is a $N\times M$ real matrix where each column ...
7
votes
1answer
229 views

$\max\min$ problem

Suppose there is a real valued function $f(x,y)$ where $x,y$ are vector variables $x\in\mathbb{R}^n$ $y\in\mathbb{R}^m$. Moreover $f$ is strictly/concave in $x$ and strictly/convex in $y$. $f$ is also ...
2
votes
1answer
264 views

Does the uniqueness of solutions to convex optimization with linear constraints hold in n>3 dimensions?

This is a repost of an earlier question, where I think I was not clear enough in what I was asking: I am examining the following optimization problem, for which I would like to know if, when a ...
2
votes
0answers
41 views

Calculating second derivative of $g(\alpha) = f(\textbf{y}(\alpha))$

I'm having problems with the second derivative of the function $g(\alpha) = f(\textbf{y}(\alpha))$ (which I will define more precisely below). I tried calculating it myself, could anyone just simply ...
1
vote
1answer
31 views

Non linear programming please HELP

Hey guys, I never do that but just found out that I have an assignment due in a few hours..thought it was for later, any help/solutions for this one?
2
votes
0answers
84 views

the objective function $\|F\|_F^2$ is quasiconvex in the optimization?why?

I have read a paper, but I can not understand one optimization thoroughly.Generally, Frobenius norm of one matrix, $\|F\|_F^2$, as the objective function is convex, so we can resolve it not using the ...
0
votes
1answer
214 views

How to get the closed form solution of a non-convex optimization problem?

I want to know if there is a closed form expression for the optimal objective function? How can I get it, if it does exist? Condition: $h,f\in \mathbb{C}^{N\times1}, \epsilon > 0 $. $\max \ \ ...
1
vote
0answers
45 views

how to minimize this convex function?

$x_i$ and $y_j$ are variables. I intend to minimize this function and obtain the optimal value of $x$ and $y$: $\begin{align} ...
0
votes
1answer
246 views

maximizing a function of a positive semi-definite matrix with bounded trace

I need to maximize a function $f(A)$ where $A$. With the constraints that $A$ is positive definite and has a trace $tr(A) \leq K$. $tr(A)=K$ will work for my problem too. I can differentiate towards ...
1
vote
0answers
38 views

On solving non-linear programming problem and the relevant software

I have a non-linear programming problem, in which all the inequality is linear and only the optimization goal is in a non-linear form. The problem is as following. $x_j$ is the variables and $a_{k,j}$ ...
1
vote
0answers
159 views

Distinction between linear and nonlinear model

[I have already asked this question on CrossValidated but until now received no answer] I have read some explanations about the properties of linear vs nonlinear models, but still I am sometimes not ...
0
votes
1answer
68 views

Approximating the optimal value of a function involving a Gaussian integral

Consider the following function $$ f(\lambda) = \alpha (1+\lambda^2) + (1-\alpha)2\int_\lambda^\infty (x-\lambda)^2 \phi(x) dx $$ where $\alpha \in (0,1)$ and $\phi$ is the standard normal probability ...
1
vote
0answers
88 views

Methods to minimise multilinear functions with trilinear, quad-linear and higher-linear terms?

My goal is to minimize functions such as $$f_1(\mathbf{p})=p_1p_3p_7+p_1p_4p_7+p_2p_3p_7+p_2p_4p_7-p_1p_3p_5p_6-p_1p_4p_5p_6-p_2p_3p_5p_6-p_2p_4p_5p_6$$ and ...
0
votes
1answer
51 views

how to solve this optimization problem with functions?

Suppose $\theta\in[0,1]$ and $s(\cdot)$ is strictly increasing in $\theta$ with $0\leq(0)<s(1)\leq1$ and $s(0)+s(1)>1$ How could I solve the program, $\max_{s(\cdot)\in ...
1
vote
0answers
65 views

Linear Complementarity Problem - multiple solutions, which one will it find?

If I have a inequality constrained system: w = Mz + q <= 0, z<=0, z^T w = 0 that for some given properties M and ...
1
vote
0answers
146 views

Dimension analysis and explaining the $\varepsilon$

Reference of this post (page no 6 from equation 39) The time averaged total energy, $\bar E$, has the following $\varepsilon$ expansion in $D$ dimension: \begin{equation} ...
0
votes
1answer
304 views

optimization of a non-differentiable, component-wise step function

I would like to estimate the (local) minimum of a function $c:R^N \mapsto R^+$ where: $c$ is only differentiable almost everywhere, there exists a component $j$, such that $\frac{\partial ...
3
votes
1answer
498 views

Optimizing trigonometric and nonlinear functions

First, Please, keep in mind that I'm a programmer not mathematician, and I have a fair mathematical background. I used optimization in Java to fit some observations to a trigonometric function, I ...
3
votes
2answers
165 views

Optimization for constrained problem

I'm reading about Lagrange multipliers from a Pattern recognition book appendix and on one point the following is stated: $\begin{align} ...
1
vote
0answers
286 views

iterative method to solve nonlinear equations

I'd like to know whether there are any methods like the Gauss-Seidel method to solve nonlinear equations. For example, I'd like to solve $f(\textbf x) = 0$, where $f(\textbf x)$ is a nonlinear ...
0
votes
1answer
56 views

Approaches to fitting noisy oscillatory data?

I have observations $\hat{f}$ from data at points $\mathbf{x}=\{x_1,\ldots,x_N\}$, that is modeled as a known oscillatory form $f(k\ x)$ (for example, the sinc function), where $k$ controls the ...
2
votes
2answers
2k views

Which optimization algorithm converges faster?

everyone. I'm having a large scale unconstrained optimization problem. If I treat the unconstrained problem as a constrained problem with infinity constraints, I should be able to use both the ...
0
votes
1answer
46 views

Directive on Dimensionality Reduction

I have a data set (24 data records) which is in $\mathbb{R}^{13}$ and I need to project it to a lower dimension (at least to $\mathbb{R}^{3}$). My objective of the dimensionality reduction is to ...
1
vote
1answer
67 views

A minimization problem

Define $$L(w,u)=\frac{1}{2}\|w-u\|^2+\beta \left\|\frac{w}{x}\right\|,~w,u\in \Bbb{R}^n$$ where $$\frac{w}{x}=\left(\frac{w_1}{x_1},\ldots, \frac{w_n}{x_n}\right)$$ $$\|x\|=\sqrt{x_1^2+\cdots+x_n^2}$$ ...
1
vote
1answer
48 views

A minimization problem [duplicate]

Define $$L(w,u)=\frac{1}{2}\|w-u\|^2+\beta \|\frac{w}{x}\|,~w,u\in R^n$$ where $$\frac{w}{x}=(\frac{w_1}{x_1},\dots, \frac{w_n}{x_n})$$ $$\|x\|=\sqrt{x_1^2+\cdots+x_n^2}$$ Given $u$, $x$ and $\beta$, ...
1
vote
1answer
1k views

Optimization of a function of two variables.

Suppose we have to minimize a function $f(\mathbf x,\mathbf y)$ where $\mathbf x$, $\mathbf y$ are vectors in Euclidean space. The function is convex in $\mathbf y$ when $\mathbf x$ is kept constant ...