# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

471 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
22 views

### need difficult 2-3dim objective functions to optimize, by algorithm

for teaching purposes, I am looking for continuous compact functions defined over one or two variables that are deliberately chosen to illustrate how optimization algorithms can run into difficulties, ...
23 views

### Numerical analysis: Chebyshev coefficient representation error.

I am unsure if numerical analysis questions are suitable for this forum, but I have nowhere else to ask, so if this question is inappropriate, tell me and I will delete it. If $x_k$ are the Chebyshev ...
226 views

I often read about the importance of momentum in machine learning, namely, in neural networks. And as the partial derivative of the cost function w.r.t. to the weights gives us gradient descent, ...
84 views

### Confusion about Optimization partial derivative

I am confuse about maximization of a smooth, well behaved objective function $f(x,y, z)$ subject to the constraint that: $0\leq x \leq 1$ $y+z \leq 200$ $y,z$ are positive. The function $f(x,y,z)$...
37 views

### B-spline surfaces fitting references

I am looking for reference papers / publications regarding b-spline / NURBS surfaces fitting (I think I have noticed NURBS fitting is not very widespread so I guess most references will be related to ...
215 views

### Stochastic optimization vs stochastic programming

How should I think about the differences between stochastic optimization (SO) and stochastic programming (SP)? From Wikipedia, it seems that SO is a framework that uses randomness to solve a pre-...
78 views

### How to choose two diagonal matrices minimizing the condition number

I have a matrix $A \in R^{n×n}$. I would like to choose two diagonal matrices $D_1,D_2 \in R^{n×n}$ such that $\text{cond}(D_1AD_2)$ should be minimal. How to provide such diagonal matrices?
95 views

### or Minimizing a sum of a product

I am trying to minimize a function and I was hoping that someone can point me in the right direction. I have tried using Lagrangian multipliers and experimenting with simple cases, but nothing has ...
51 views

### How does one solve non-linear systems of equations with only finite degree polynomial terms over the reals?

Consider a general polynomial non-linear system of equations as follows over the reals: $$\begin{array}{c} f_1(x_1, \cdots, x_D) = 0 \\ \vdots \\ f_N(x_1, \cdots, x_D) = 0 \\ \end{array}$$ note ...
62 views

### Reducing sensitivity to initial guess for this nonconvex optimization problem

I'll start by formulating my problem. I am given a point and a plane. I am allowed to apply gains to the point such that it touches the plane, and is closest in 2 norm to the original point. The gains ...
164 views

### SVM classifier manual implementation differs from scikit

I'm trying to manually implement the scikit learn basic SVM classifier using the Gram kernel matrix $K$. The mathematic formulation is the following: \begin{align} \min_{\alpha\in\mathbb{R}^n} \...
64 views

### optimal orthogonal matrix in L1 sense

I want to find an orthogonal matrix $O\in SO(n)$ such that $\|Y - OX \|_1$ is minimized, where X and Y are matrices (of appropriate sizes). I know that there is a solution to this problem using SVD ...
87 views

### $\ell_1$ minimization with quadratic constraint

Is there a tractable solution to the optimization problem $$\min \|Ax\|_1 \mbox{ such that } \|x\|_2^2 = 1?$$ Because of the non-convexity of the equality constraint, it seems like this is hard. (In ...
72 views

### Explicit least-squares method for horizontal shifts of a function

I have a sequence of $N$ strictly positive real values $y_n$. They form some kind of peak; for simplicity, let's assume $f(x, \mu) = A \exp^{-(x-\mu)^2}$ is the shape, with $A$ and $\mu$ real (in the ...
892 views

### Does converting an inequality constraint to an equality one have any major impact on an optimization solver?

In an optimization problem, I have an inequality constraint, say $\begin{array}{c} {\min\limits_x~} c(x)\\ {s.t.~}g(x)\le 0 \end{array}$ The function $g(x)$ in general is unknown. So, numerical ...
110 views

### Index of a stationary point of constrained optimization

For an unconstrained optimization problem with objective function $F(x,y,z)$ the index of a stationary point is well-defined: If $(x^*, y^*, z^*)$ is a point where the gradient of $F(x,y,z)$ vanishes, ...
77 views

### Isotonic regression like

I have 2 ordered sets $$X=\{X_1<\dots<X_n\}$$ and $$Y=\{Y_1<\dots<Y_m\}$$ with $X_1<Y_1$ and and $X_n<Y_m$. I wish to approximate an increasing continuous function $g$ by piecewise ...
2k views

303 views

### Good convergence criterion for stochastic optimization?

This is a question that has bothered me quite long, as I have faced it many different optimization and equation solving problems. The basic idea is that one wishes to minimize $F(x)$ and has one ...