# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

481 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
93 views

### Recommendation for intro to geometric integrators?

Explicit Request Looking for book or lecture note recommendations on numerical optimization that (ideally) have the following: Emphasis on geometric and physical intuition Emphasis on symplectic ...
4k views

### Jacobian of exponential mapping in SO3/SE3

Following this post Jacobian matrix of the Rodrigues' formula (exponential map) What if I really need the Jacobian of the exponential mapping function in $\omega \neq 0$? Basically, I want to ...
52 views

### What is known about optimizing a function whose evaluation cost is variable?

Let's imagine I have a function $f(x)$ whose evaluation cost (monetary or in time) is not constant: for instance, evaluating $f(x)$ costs $c(x)\in\mathbb{R}$ for a known function $c$. I am not ...
508 views

I am curious about divergence conditions of the Gradient Descent method. In the reading that I have done I have only ever seen it mentioned that this method diverges when $\alpha$ is too large. It ...
67 views

Let $f(x)$ be a function on $[0,+\infty)$. I want to solve the following functional problem: $\min L(f)=\int_0^{\infty} (f'(x))^2\mathrm dx$ subject to $f(0)=a$ $f(x)\ge 0,\forall x$ $\int_0^\... 0answers 286 views ### Is there a name for this modified Newton method, and prove the convergence. I can't seem to find the name of this method anywhere in literature, but it has appeared in my assignment. Modified Newton Method. Let$f\in C^2$, convex,$\mathbb{R}^n\to\mathbb{R}$. The ... 0answers 176 views ### Qualities of Projected Gradient Methods Consider the following constrained minimization problem:$ min_{x \in X} \ f(x) $where$ X \subset \Bbb{R}^{n} $is a nonempty closed convex set and f is continuously diferentiable. I'm ... 0answers 739 views ### Real-time linear programming I'm going to implement in C a light-weight embedded LP solver for a production system. I need to be able to sequentially solve a series of (possibly unrelated) linear programs with ~6-60 variables and ... 2answers 118 views ### Efficient algorithm for determining whether value of convex optimization program is below some value? Let$X$be a convex subset of$\mathbb{R}^N$, let$c \in \mathbb{R}^N$. I want to know whether $$\min_{x \in X} x^\top c < 0.$$ Obviously, I can (efficiently, with standard software) evaluate ... 0answers 57 views ### How can I apply backpropagation with matrix algebra? - Deep learning Deep learning and backpropagation is taught out very badly and is often looks like a mess, according to me. So I want to start with a simple example about how to use backpropagation: Assume that we ... 0answers 63 views ### Adding identity to invert matrix I'm looking into algorithm implementation which is essentially linear regression: $$\|Ax - b\| \rightarrow \min$$ Matrix A and vector b are estimated using data, then we do$A^{-1}b$to find x. But ... 0answers 38 views ### LMI-based versus standard form semidefinite programs In the context of semidefinite programming (SDP), under what conditions is it preferable to formulate and solve an LMI-based SDP rather than an equivalent standard form SDP? I have been told that ... 0answers 136 views ### Verify if my idea is correct Hi guys I have been reading Numerical optimization by Nocedal. While I was on the chapter on quasi- Newton methods it was mentioned that$g_{k+1}^Ts_k=0$where$g_k = Ax_k -b^Tx_k$if we use exact ... 0answers 39 views ### Optimizing intervals in piecewise function I am not sure what the relevant tags are (or how to best described the problem). I have a piecewise function where for a particular interval element there is a known functional form (e.g. for ... 1answer 287 views ### A good book on programming for numerical optimization I was hoping someone could recommend a good book on programming for numerical optimization--including lots of code examples. I am reading Nocedal and Wright, which is great. One of the recommendations ... 0answers 86 views ### Infinite dimensional convex linear optimisation problem I have the following problem: The functions$a_i(x) > 0$and$b_i(x) > 0$for$x\in I \subset \mathbb{R}$,$I$compact, and$i=1,\ldots,n$are given. The objective is to find functions$f_i(x)$... 0answers 84 views ### Maximize rank of Gramian kernel matrix Suppose we have a data matrix$X \in \mathbb{R}^{m\times n}$, where$m\gg n$. This matrix represents a series of measurements of a physical system, where each column represent a single measurement. ... 0answers 423 views ### Gradient descent with box constraints and possible non-convex function. Hope you are well. I am working on an optimization problem, quadratic (see below). Of the 4 variables there are but 2 that have a negativity constraint. Am I correct to say that gradient descent is ... 0answers 183 views ### ADMM fails to converge on convex problem. Are there any tricks of trade for application? Convex Problem I am trying to solve the semidefinite program:$\min y$(Objective, 0) subject to$y\geq0$(Nonnegative, 1)$y I + \Sigma_0 = \sum^{n}_{i=1} x_i \Sigma_i + Z$(Linear Equality,2) ... 0answers 357 views ### Generalized gradient descent with constraints In order to find the local minima of a scalar function$f(x)$, where$x \in \mathbb{R}^N$, I know we can use the projected gradient descent method if I want to ensure a constraint$x\in C$: $$y_{k+1}=... 0answers 151 views ### Efficient algorithm for lower-bound least squares. We have: A \in \mathbb{R}^{n \times m} with independent columns, y \in \mathbb{R}^n. Moreover, n \gg m. Consider the following problem, where the inequality is elementwise:$$x^{\star} := \arg\... 0answers 38 views ### Numerical “helper solver” to solve polynomial matrix equation system? I have noticed that when solving the following matrix-polynomial: $$\sum_{k=0}^N{\bf C}_k{\bf T}^k = {\bf 0} \hspace{0.6cm} \text{ s.t. } \hspace{0.6cm} {\bf C}_k,{\bf T} \in\mathbb{R}^{{M\times M}}$$... 0answers 107 views ### how to find the the maximum of an implicit function I have an implicit function and I would like to find the value of$h$that maximizes$R$, i.e, I want to find$h$that satisfies$\frac{\partial R}{\partial h} = 0$. The function is,$C=\frac{A}{1+\...
221 views

Let's consider smooth and convex minimization problem, i.e. $min f(x)$, where $f$ is not necessarily a quadratic function. If measured by iterations, Accelerated Gradient Descend (AGD) has $O(1/T^2)$...
933 views

30 views

### Globalized BFGS (Quasi-Newton method) condition

I didn't find any information on the internet about the globalized $\textit{BFGS}$ method. Wikipedia only talks about the normal BFGS, without this ...
53 views

### A “divide-and-conquer” iterative procedure for minimizing a sum of convex functions

For simplicity, let's assume $f_i: \mathbb{R} \to \mathbb{R}$ is convex and define $$g(x) := \sum_{i=1}^{n}f_i(x)$$ Suppose we want to compute $$x^* := \arg\min_{x \in \mathbb{R}} g(x)$$ ...
11 views

### Finding the domain shape, where the fundamental Helmholtz Eigenmode has certain properties at a boundary

The following is a problem about finding the shape of a domain such that the fundamental Helmholtz eigenmode has certain properties at the boundary of the domain. I search for an analytical or ...
68 views

### How do I pack 2D objects into an arbitrary shape?

I've heard of packing problems in general, and I've seen a couple questions on this site that address specific cases (e.g. how to pack 45-45-90 triangles into an arbitrary shape), but after a search I ...
88 views

### Using Coordinate Descent on Projected Space

My goal is to maximize an objective function using coordinate descent over a 3-dimensional vector. In the simple case the domain over which I am maximizing is defined as follows: $X \in \mathcal{X}$ ...
Consider the following convex optimization problem in vectors $x$ and $y$ $$\begin{array}{ll} \text{minimize} & f(x,y)\\ \text{subject to} & g_1 (x)\le 0\\ & g_2 (y)\le 0\end{array}$$ ...