# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

475 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0answers
63 views

### Finding a point with maximum distance from a given point in a polyhedron

We are given the polyhedron $X=\{x:Ax\le b,x\ge 0\}$ and the point $y\in X$. We want to find a point $x \in X$ such that $d(x,y)$ is maximized. The function $d(x,y)$ represents the distance between ...
0answers
46 views

### Monge Ampere Numerical Analysis

This is an example of the Monge-Ampere equation from "C0 penalty methods for the fully nonlinear Monge-Ampere equation, Mathematics of Computation, 80:1979-1995, 2011." by S.C. Brenner, T. Gudi, M. ...
0answers
66 views

### Optimality guarantees of SGD convergence in Geometric Programming

What guarantees of optimality do we get when minimizing with Stochastic Gradient Descent a problem in its original formulation, after showing that it is a Geometric Programming instance (i.e. can be ...
0answers
22 views

### need difficult 2-3dim objective functions to optimize, by algorithm

for teaching purposes, I am looking for continuous compact functions defined over one or two variables that are deliberately chosen to illustrate how optimization algorithms can run into difficulties, ...
0answers
23 views

### Numerical analysis: Chebyshev coefficient representation error.

I am unsure if numerical analysis questions are suitable for this forum, but I have nowhere else to ask, so if this question is inappropriate, tell me and I will delete it. If $x_k$ are the Chebyshev ...
1answer
248 views

### Momentum in gradient descent

I often read about the importance of momentum in machine learning, namely, in neural networks. And as the partial derivative of the cost function w.r.t. to the weights gives us gradient descent, ...
0answers
85 views

### Confusion about Optimization partial derivative

I am confuse about maximization of a smooth, well behaved objective function $f(x,y, z)$ subject to the constraint that: $0\leq x \leq 1$ $y+z \leq 200$ $y,z$ are positive. The function $f(x,y,z)$...
0answers
37 views

### B-spline surfaces fitting references

I am looking for reference papers / publications regarding b-spline / NURBS surfaces fitting (I think I have noticed NURBS fitting is not very widespread so I guess most references will be related to ...
0answers
237 views

### Stochastic optimization vs stochastic programming

How should I think about the differences between stochastic optimization (SO) and stochastic programming (SP)? From Wikipedia, it seems that SO is a framework that uses randomness to solve a pre-...
2answers
82 views

### How to choose two diagonal matrices minimizing the condition number

I have a matrix $A \in R^{n×n}$. I would like to choose two diagonal matrices $D_1,D_2 \in R^{n×n}$ such that $\text{cond}(D_1AD_2)$ should be minimal. How to provide such diagonal matrices?
0answers
98 views

### or Minimizing a sum of a product

I am trying to minimize a function and I was hoping that someone can point me in the right direction. I have tried using Lagrangian multipliers and experimenting with simple cases, but nothing has ...
0answers
51 views

### How does one solve non-linear systems of equations with only finite degree polynomial terms over the reals?

Consider a general polynomial non-linear system of equations as follows over the reals: $$\begin{array}{c} f_1(x_1, \cdots, x_D) = 0 \\ \vdots \\ f_N(x_1, \cdots, x_D) = 0 \\ \end{array}$$ note ...
0answers
63 views

### Reducing sensitivity to initial guess for this nonconvex optimization problem

I'll start by formulating my problem. I am given a point and a plane. I am allowed to apply gains to the point such that it touches the plane, and is closest in 2 norm to the original point. The gains ...
0answers
169 views

### SVM classifier manual implementation differs from scikit

I'm trying to manually implement the scikit learn basic SVM classifier using the Gram kernel matrix $K$. The mathematic formulation is the following: \begin{align} \min_{\alpha\in\mathbb{R}^n} \...
0answers
64 views

### optimal orthogonal matrix in L1 sense

I want to find an orthogonal matrix $O\in SO(n)$ such that $\|Y - OX \|_1$ is minimized, where X and Y are matrices (of appropriate sizes). I know that there is a solution to this problem using SVD ...
0answers
87 views

### $\ell_1$ minimization with quadratic constraint

Is there a tractable solution to the optimization problem $$\min \|Ax\|_1 \mbox{ such that } \|x\|_2^2 = 1?$$ Because of the non-convexity of the equality constraint, it seems like this is hard. (In ...
1answer
75 views

### Explicit least-squares method for horizontal shifts of a function

I have a sequence of $N$ strictly positive real values $y_n$. They form some kind of peak; for simplicity, let's assume $f(x, \mu) = A \exp^{-(x-\mu)^2}$ is the shape, with $A$ and $\mu$ real (in the ...
1answer
909 views

### Does converting an inequality constraint to an equality one have any major impact on an optimization solver?

In an optimization problem, I have an inequality constraint, say $\begin{array}{c} {\min\limits_x~} c(x)\\ {s.t.~}g(x)\le 0 \end{array}$ The function $g(x)$ in general is unknown. So, numerical ...
0answers
110 views

### Index of a stationary point of constrained optimization

For an unconstrained optimization problem with objective function $F(x,y,z)$ the index of a stationary point is well-defined: If $(x^*, y^*, z^*)$ is a point where the gradient of $F(x,y,z)$ vanishes, ...
0answers
77 views

### Isotonic regression like

I have 2 ordered sets $$X=\{X_1<\dots<X_n\}$$ and $$Y=\{Y_1<\dots<Y_m\}$$ with $X_1<Y_1$ and and $X_n<Y_m$. I wish to approximate an increasing continuous function $g$ by piecewise ...
1answer
2k views

1answer
310 views

### Good convergence criterion for stochastic optimization?

This is a question that has bothered me quite long, as I have faced it many different optimization and equation solving problems. The basic idea is that one wishes to minimize $F(x)$ and has one ...
1answer
113 views

### why not handle box-constraints with a transformation

I have a question that I've always wondered about concerning the "L-BFGS-B" algorithm. I am not familar with the details of the algorithm except for the fact that it optimizes a non-linear function ...
0answers
45 views

0answers
40 views

### Derive Hessian inverse update using Sherman-Morrison in Quasi Newton Method

From Nocedal's Numerical Optimization book, the Hessian approximation equation is given using Symmetric Rank 1 (SR1) formula as follows: B_{k+1} = B_k + \frac{(y_k - B_ks_k)(y_k - B_ks_k)}{(y_k - ...