# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

911 questions
Filter by
Sorted by
Tagged with
23 views

### Showing a Chebyshev set

I want to show that $\{1,e^{ix},...,e^{(n-1)x} \}$ is a Chebyshev Set on $(0,2\pi]$. Now I know that $\{1,x,...,x^n \}$ is one and that $e^{ix}$ is injective on $(0,2\pi]$. But how do I show that if I ...
26 views

### What are possible ways to update the Hessian if the calculated gradient is negative in BFGS algorithm (Quasi-Newton method)?

When applying the globalized BFGS algorithm (Quasi-Newton Method, optimization, minimization) to approximate the minimum of a function using the Quasi-Newton-Method, sometimes one can get a negative ...
46 views

### When is the simplex method slower than the ellipsoid algorithm?

In an undergrad class on linear programming, we learned about the simplex and ellipsoid methods for solving linear programs (LPs). I know that the simplex method is generally faster than the ellipsoid ...
1k views

### Finding good approximation for $x^{1/2.4}$

I would like to a good (8 bits accuracy) approximation for $x^{1/2.4}$ in the range $[0, 1]$. This transform is used for converting linear intensities to SRGB compressed values, so it's important that ...
43 views

### $\lambda_{max}$ in trust region method: Leveberg-Marquardt algorithm

I am learning levenberg-marquardt algorithm and in the process implementing the same. I am comfortable with the $Jacobian$, $Hessian$ and step size computation. For trust region implementation, I have ...
13 views

### multi-objective optimization test function explanation

i took this image from a paper that describes a multi-objective optimization algorithm where UF1 is a multi-objective function to optimize. can you explain to me what J1, J2 variables and the second ...
54 views

I am reading Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, where Boyd claimed that ADMM is an algorithm that is intended to blend the ...
31 views

### Find a suitable zero equation to solve the optimization problem $\min_{x \in \mathbb{R}^N} f(x)$

Suppose we have an optimization problem for this general form of $f: \mathbb{R}^N \rightarrow \mathbb{R}$ $$\min_{x \in \mathbb{R}^N} f(x)$$ and this problem is solvable. How could I construct a ...
405 views

### What is the standard SDP form of this eigenvalue optimization?

The following are two pictures in a lecture note: I know how to formulate this problem into the second to last problem, but I am confused about how to write this problem into the standard SDP problem....
25 views

28 views

19 views

### How can I prove $\lambda_k^* = \frac{-g_k^T u_k}{u^T_k A u_k}$ in the conjugate gradient method?

To break down the formula, This only applies to quadratic functions, $q(x) = \frac12 x^TAx+bx+c$ $g_k$ is the first derivative at the point $x_k$ $A$ is found from the quadratic function, or, I think,...
37 views

### Online Non-convex optimization

Can stochastic gradient descent be used for online non-convex optimization? If not, what are the suitable algorithms?
46 views

### Optimise allocation to minimise variance

Background I am trying to allocate customers $C_i$ to financial advisers $P_j$. Each customer has a policy value $x_i$. I'm assuming that the number of customers ($n$) allocated to each adviser is ...
11 views

### What is the advantage of using KKT-CQ over LI-CQ for first-order KKT necessary condition

In the notes I follow, the author uses the KKT-CQ for the first-order necessary KKT conditions, then impose an additional constraint qualification which is the LICQ for the second-order conditions. ...
21 views

### What is the fault with Fritz John necessary condition for finding a local minimum for a general NLPP

The author says: It is also possible that, at some feasible point $x$, the FJ conditions are satisfied with Lagrange multiplier associated with the objective function $u_0 = 0$. In those ...
45 views

40 views

37 views

### What point will ADMM converge to ? A feasible point or a stationary point or local optima or global optima?

In Boyd's great ADMM paper Section 3.2.1, ''Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers'', it says that as iteration index $k\to \infty$, ...
38 views

### Numerical methods for solving the inverse Weierstrass transform

I have a measurement technique that results in a frequency distribution of the results. Let's say x is the parameter of interest, the method gives you a distribution F(x). I have the idea that the ...
19 views

### Analysis of optimization procedure for two equivalent optimization problems

I come up with this question. Assume the optimization problem is: $$argmin_w\|\hat{y}(w)-y\|^2$$ Now suppose there is a overcomplete set for the representation of $\hat{y}$ and $y$, this set has lots ...
Problem: Find (numerically) minimum of a function $f=f(x_1,...,x_M)$, where $M \in \mathbb{N}$ - a fixed number (often large), and $x_i \in [a,b],\forall i$. Function $f$ is complicated, and to ...