# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

911 questions
Filter by
Sorted by
Tagged with
1answer
23 views

### Showing a Chebyshev set

I want to show that $\{1,e^{ix},...,e^{(n-1)x} \}$ is a Chebyshev Set on $(0,2\pi]$. Now I know that $\{1,x,...,x^n \}$ is one and that $e^{ix}$ is injective on $(0,2\pi]$. But how do I show that if I ...
0answers
26 views

### What are possible ways to update the Hessian if the calculated gradient is negative in BFGS algorithm (Quasi-Newton method)?

When applying the globalized BFGS algorithm (Quasi-Newton Method, optimization, minimization) to approximate the minimum of a function using the Quasi-Newton-Method, sometimes one can get a negative ...
0answers
46 views

### When is the simplex method slower than the ellipsoid algorithm?

In an undergrad class on linear programming, we learned about the simplex and ellipsoid methods for solving linear programs (LPs). I know that the simplex method is generally faster than the ellipsoid ...
4answers
1k views

### Finding good approximation for $x^{1/2.4}$

I would like to a good (8 bits accuracy) approximation for $x^{1/2.4}$ in the range $[0, 1]$. This transform is used for converting linear intensities to SRGB compressed values, so it's important that ...
0answers
43 views

### $\lambda_{max}$ in trust region method: Leveberg-Marquardt algorithm

I am learning levenberg-marquardt algorithm and in the process implementing the same. I am comfortable with the $Jacobian$, $Hessian$ and step size computation. For trust region implementation, I have ...
0answers
13 views

### multi-objective optimization test function explanation

i took this image from a paper that describes a multi-objective optimization algorithm where UF1 is a multi-objective function to optimize. can you explain to me what J1, J2 variables and the second ...
0answers
54 views

### Block Separability in ADMM

I am reading Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, where Boyd claimed that ADMM is an algorithm that is intended to blend the ...
1answer
31 views

### Find a suitable zero equation to solve the optimization problem $\min_{x \in \mathbb{R}^N} f(x)$

Suppose we have an optimization problem for this general form of $f: \mathbb{R}^N \rightarrow \mathbb{R}$ $$\min_{x \in \mathbb{R}^N} f(x)$$ and this problem is solvable. How could I construct a ...
1answer
405 views

### What is the standard SDP form of this eigenvalue optimization?

The following are two pictures in a lecture note: I know how to formulate this problem into the second to last problem, but I am confused about how to write this problem into the standard SDP problem....
1answer
25 views

1answer
28 views

0answers
19 views

### How can I prove $\lambda_k^* = \frac{-g_k^T u_k}{u^T_k A u_k}$ in the conjugate gradient method?

To break down the formula, This only applies to quadratic functions, $q(x) = \frac12 x^TAx+bx+c$ $g_k$ is the first derivative at the point $x_k$ $A$ is found from the quadratic function, or, I think,...
0answers
37 views

### Online Non-convex optimization

Can stochastic gradient descent be used for online non-convex optimization? If not, what are the suitable algorithms?
1answer
46 views

### Optimise allocation to minimise variance

Background I am trying to allocate customers $C_i$ to financial advisers $P_j$. Each customer has a policy value $x_i$. I'm assuming that the number of customers ($n$) allocated to each adviser is ...
0answers
11 views

### What is the advantage of using KKT-CQ over LI-CQ for first-order KKT necessary condition

In the notes I follow, the author uses the KKT-CQ for the first-order necessary KKT conditions, then impose an additional constraint qualification which is the LICQ for the second-order conditions. ...
0answers
21 views

### What is the fault with Fritz John necessary condition for finding a local minimum for a general NLPP

The author says: It is also possible that, at some feasible point $x$, the FJ conditions are satisfied with Lagrange multiplier associated with the objective function $u_0 = 0$. In those ...
1answer
45 views

0answers
40 views

1answer
37 views

### What point will ADMM converge to ? A feasible point or a stationary point or local optima or global optima?

In Boyd's great ADMM paper Section 3.2.1, ''Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers'', it says that as iteration index $k\to \infty$, ...
1answer
38 views

### Numerical methods for solving the inverse Weierstrass transform

I have a measurement technique that results in a frequency distribution of the results. Let's say x is the parameter of interest, the method gives you a distribution F(x). I have the idea that the ...
1answer
19 views

### Analysis of optimization procedure for two equivalent optimization problems

I come up with this question. Assume the optimization problem is: $$argmin_w\|\hat{y}(w)-y\|^2$$ Now suppose there is a overcomplete set for the representation of $\hat{y}$ and $y$, this set has lots ...
0answers
34 views

### Algorithm to find minimum of a multivariate function

Problem: Find (numerically) minimum of a function $f=f(x_1,...,x_M)$, where $M \in \mathbb{N}$ - a fixed number (often large), and $x_i \in [a,b],\forall i$. Function $f$ is complicated, and to ...
0answers
34 views

### Globalized Newton method for minimizing a specific functional: Convergence?

I'm currently working on a generalized p-Laplace equation: \begin{align} \label{DP} \begin{cases} \text{div} (\sigma |\nabla u|^{p-2} \nabla u) = f &\quad \text{in } \Omega\\ u = g &\quad \...