# Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

892 questions
Filter by
Sorted by
Tagged with
684 views

56 views

### Adding identity to invert matrix

I'm looking into algorithm implementation which is essentially linear regression: $$\|Ax - b\| \rightarrow \min$$ Matrix A and vector b are estimated using data, then we do $A^{-1}b$ to find x. But ...
29 views

### Extracting diagonal of $J^TJ$ via automatic differentiation like techniques

First of all please, let me know if this question is more suited for scicomp.stackexchange.com or or.stackexchange.com, and ...
28 views

### Can we always find a proper step size?

In convex optimization, if we know the gradient of a function $f(x)$, then is it true that we could always find a way to determine a proper step size in the gradient descent method? When I say "proper"...
34 views

### LMI-based versus standard form semidefinite programs

In the context of semidefinite programming (SDP), under what conditions is it preferable to formulate and solve an LMI-based SDP rather than an equivalent standard form SDP? I have been told that ...
1k views

### Direct multiple shooting (numerical optimal control)

please, Iam currently implementing direct multiple shooting method* and I need one simple but fundamental concept answered: When I want to provide not only objective funtion value (result of ODE ...
38 views

### Positive semidefinite inequality

Let $\mathbf{A}\in\mathbb{C}^{m\times m}$, and $\mathbf{B}\in\mathbb{H}^{m\times m}$ be an $m$-dimension Hermitian matrix, solve $\theta$ that satisfies the condition \begin{equation*} e^{\jmath\theta}...
14 views

### Optimal approximation of nonlinear probability density function by piecewise constant density

Given a nonlinear probability density function F, the problem is to estimate F using histogram over a partition with N intervals. I have tried to realise this with MATLAB function fmincon, but it ...
46 views

### What is known about optimizing a function whose evaluation cost is variable?

Let's imagine I have a function $f(x)$ whose evaluation cost (monetary or in time) is not constant: for instance, evaluating $f(x)$ costs $c(x)\in\mathbb{R}$ for a known function $c$. I am not ...
134 views

### Verify if my idea is correct

Hi guys I have been reading Numerical optimization by Nocedal. While I was on the chapter on quasi- Newton methods it was mentioned that $g_{k+1}^Ts_k=0$ where $g_k = Ax_k -b^Tx_k$ if we use exact ...
100 views

38 views

79 views

22 views

### Gradient-based interpretation of the simplex algorithm

The simplex algorithm iterates from vertex to vertex of the convex polytope that bounds the feasible region of the constrained optimization problem, such that each iteration of the algorithm moves ...
44 views

### Understanding the steps of Karmarkar's algorithm

I am working through Karmarkar's seminal paper  and came across something I didn't quite understand. In section 2.3, Description of the Algorithm, he explains how to calculate the next point. The ...
892 views

### Does converting an inequality constraint to an equality one have any major impact on an optimization solver?

In an optimization problem, I have an inequality constraint, say $\begin{array}{c} {\min\limits_x~} c(x)\\ {s.t.~}g(x)\le 0 \end{array}$ The function $g(x)$ in general is unknown. So, numerical ...
2k views

28 views

### Divergence criteria of Secant method on $\arctan(x)$?

I want to make sure I understand when the secant method will not converge as compared to the Newton's method. When I look at $\arctan(x)$ and try to determine the initial guesses for which it will ...
I'm trying to find a numerical solution to the following optimization problem $$\text{maximize } f(M) = \frac{1}{2} \log \det(M) + \log \sum_{i=1}^n \exp \left\{ - \frac{1}{2} x_i^T M x_i + a_i \... 1answer 20 views ### Prove that \xi_{k+1} = (-1)^{k+1}(\alpha_0 \times \alpha_1 \times \dots \times \alpha_k) is the (k+1) coefficient of p_k I was given the following question as part of a homework assignment. Any help would be greatly appreciated! The following image shows the steps of a preliminary version of the conjugate gradient ... 1answer 47 views ### “Rectangular” Cholesky decomposition of lower dimension Given a symmetric PSD matrix A \in \mathbb R^{n \times n}, we can Cholesky-decompose it into LL^T, where L \in \mathbb R^{n \times n} is lower triangular. However, we can also consider ... 0answers 25 views ### Globalized BFGS (Quasi-Newton method) condition I didn't find any information on the internet about the globalized \textit{BFGS} method. Wikipedia only talks about the normal BFGS, without this ... 2answers 64 views ### How to show equivalence between two programs? Let$$A := \left\{ (x_1,x_2,x_3) \in \mathbb{R}^3 \mid x_1+x_2+x_3 = 1 \right\}$$and suppose that we want to minimize a function J : \mathbb{R}^{3} \to \mathbb{R} subject to the constraint y \... 0answers 58 views ### Computational complexity of solving an SDP in CVX I have solved an SDP by the MOSEK solver of the CVX toolbox. I need to calculate the computational complexity of my algorithm. Can you help me in this regard? I would appreciate it if you can give me ... 0answers 35 views ### Understanding/Proving a theorem in Numerical Optimization by Nocedal I was reading this book specially theorem 8.4 on page 210. Suppose that a method in the Broyden class is applied to a strongly convex quadratic function f : R^n \rightarrow R, where x_0 is the ... 0answers 23 views ### constraint optimization using penalty function Let say you have following constraint optimization problem and you want to optimize it using penalty function method:$$ \min f(\mathbf{x}), \mathbf{x} \in R^{2} \\ s.t. \mathbf{a(x) = 0}, \\ \mathbf{...
In my computetional methods course we recently had an algorithm for solving $(P)$ : $\min_{x \in \mathbb{R}^n} f(x) = \frac{1}{2}x^THx + c^Tx$ subject to $a_i \leq x_i \leq b_i$ for \$i \in \...