# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

907 questions
Filter by
Sorted by
Tagged with
2answers
32 views

### A global optima: $\text{max}_{x} \frac{1}{2}\left\| X (a + b) \right\|_2^2 \ \text{s.t.} \ a^T X b \leq \delta; 0 < x \leq 1$,$X := {\rm Diag}(x)$

How to find (using any software) a global optimum for such a (non-convex) problem \begin{align} \text{maximize}_{x \in \mathbb{R}^{n \times 1}} \quad & \frac{1}{2}\left\| X (a + b) \right\|_2^2\\ ...
1answer
31 views

### Generalizing the conjugate gradient like this works?

Given $A \in \mathbb{R}^{n \times n}$, a SPD matrix, and a vector $b \in \mathbb{R}^n$, it is possible to solve the problem $$\min_x \| Ax - b\|$$ with the conjugate gradient method. Its algorithm ...
1answer
45 views

### How to optimize a function with the following constraints by using gradient descent?

I am not currently unfamiliar with a numerical optimization, so I am studying them. What I am wondering is that I'd like to optimize a certain function with the following constraints by using gradient ...
0answers
15 views

### What does “reduced-Hessian” mean? [on hold]

What does "reduced-Hessian" mean? I came across this concept in the book named "numerical optimization". Thanks a lot.
0answers
22 views

1answer
12 views

1answer
37 views

### Variational inference: Does the natural gradient follow geodesics locally?

Amari's natural gradient descent is a well-known optimisation algorithm from information geometry that is well-suited for finding optima of functionals on statistical manifolds. It consists of ...
2answers
49 views

0answers
61 views

### Adding identity to invert matrix

I'm looking into algorithm implementation which is essentially linear regression: $$\|Ax - b\| \rightarrow \min$$ Matrix A and vector b are estimated using data, then we do $A^{-1}b$ to find x. But ...
0answers
30 views

### Extracting diagonal of $J^TJ$ via automatic differentiation like techniques

First of all please, let me know if this question is more suited for scicomp.stackexchange.com or or.stackexchange.com, and ...
0answers
28 views

### Can we always find a proper step size?

In convex optimization, if we know the gradient of a function $f(x)$, then is it true that we could always find a way to determine a proper step size in the gradient descent method? When I say "proper"...
1answer
17 views

### Optimal approximation of nonlinear probability density function by piecewise constant density

Given a nonlinear probability density function F, the problem is to estimate F using histogram over a partition with N intervals. I have tried to realise this with MATLAB function fmincon, but it ...
0answers
36 views

### LMI-based versus standard form semidefinite programs

In the context of semidefinite programming (SDP), under what conditions is it preferable to formulate and solve an LMI-based SDP rather than an equivalent standard form SDP? I have been told that ...
1answer
44 views

### Positive semidefinite inequality

Let $\mathbf{A}\in\mathbb{C}^{m\times m}$, and $\mathbf{B}\in\mathbb{H}^{m\times m}$ be an $m$-dimension Hermitian matrix, solve $\theta$ that satisfies the condition \begin{equation*} e^{\jmath\theta}...
1answer
52 views

### What is known about optimizing a function whose evaluation cost is variable?

Let's imagine I have a function $f(x)$ whose evaluation cost (monetary or in time) is not constant: for instance, evaluating $f(x)$ costs $c(x)\in\mathbb{R}$ for a known function $c$. I am not ...
1answer
100 views

1answer
38 views

0answers
47 views

### Understanding the steps of Karmarkar's algorithm

I am working through Karmarkar's seminal paper  and came across something I didn't quite understand. In section 2.3, Description of the Algorithm, he explains how to calculate the next point. The ...
0answers
24 views

1answer
30 views

### Divergence criteria of Secant method on $\arctan(x)$?

I want to make sure I understand when the secant method will not converge as compared to the Newton's method. When I look at $\arctan(x)$ and try to determine the initial guesses for which it will ...
0answers
27 views

### Globalized BFGS (Quasi-Newton method) condition

I didn't find any information on the internet about the globalized $\textit{BFGS}$ method. Wikipedia only talks about the normal BFGS, without this ...
1answer
49 views

### “Rectangular” Cholesky decomposition of lower dimension

Given a symmetric PSD matrix $A \in \mathbb R^{n \times n}$, we can Cholesky-decompose it into $LL^T$, where $L \in \mathbb R^{n \times n}$ is lower triangular. However, we can also consider ...