Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

490 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
21 views

Is there a way to identify singular points in a spectrahedron without finding the entire set?

I'm a physicist working in a non-linear optimization problem that reduces to a semidefinite program (SDP). In the simple low dimensional cases we workout in the research we could identify singular ...
1
vote
0answers
30 views

Extracting diagonal of $J^TJ$ via automatic differentiation like techniques

First of all please, let me know if this question is more suited for scicomp.stackexchange.com or or.stackexchange.com, and ...
1
vote
0answers
19 views

Formula for Area of a Triangle - nodal basis function

Let T be a triangle with corners $P_1, P_2, P_3$ and the nodal basis function $\lambda_1, \lambda_2, \lambda_3$ and $\alpha, \beta, \in \mathbb{N}_0$. I want to show that $$ \int_{T}^{} \lambda_1^\...
1
vote
0answers
28 views

What are possible ways to update the Hessian if the calculated gradient is negative in BFGS algorithm (Quasi-Newton method)?

When applying the globalized BFGS algorithm (Quasi-Newton Method, optimization, minimization) to approximate the minimum of a function using the Quasi-Newton-Method, sometimes one can get a negative ...
1
vote
0answers
49 views

When is the simplex method slower than the ellipsoid algorithm?

In an undergrad class on linear programming, we learned about the simplex and ellipsoid methods for solving linear programs (LPs). I know that the simplex method is generally faster than the ellipsoid ...
1
vote
0answers
44 views

$\lambda_{max}$ in trust region method: Leveberg-Marquardt algorithm

I am learning levenberg-marquardt algorithm and in the process implementing the same. I am comfortable with the $Jacobian$, $Hessian$ and step size computation. For trust region implementation, I have ...
1
vote
0answers
42 views

Understanding/Proving a theorem in Numerical Optimization by Nocedal

I was reading this book specially theorem 8.4 on page 210. Suppose that a method in the Broyden class is applied to a strongly convex quadratic function $f : R^n \rightarrow R$, where $x_0$ is the ...
1
vote
1answer
31 views

How to use Finite Difference Method solving ODE with Boundary Value Problems?

Using these formulas, it is clear how to solve the problem: For node 1, we have the boundary value on the left side, for ex. $u(0) = 0$ and for node 2, we use the formula replacing $u''$ with $u_{i-1}...
1
vote
0answers
80 views

How to implement Finite Difference Method ODE Boundary Value Problem in Python?

I implemented the Finite Differences Method for an ODE with Boundary Value Problem. Here is the approximations I used for the FDM: And here is the balk problem: with u(0) = u(L) = 0 (attached on ...
1
vote
0answers
22 views

The notion of conflicting objective functions in multi-objective optimization

In a paper by Carlos A. Coello titled A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques he states: "Multiobjective optimization (also called multicriteria ...
1
vote
0answers
28 views

Convergence Rates of Stochastic Gradient Descent with different sample size

Given a convex function $F(x)$ to be optimized with $F(x^*)$ being the optimal value at $x^*\in\mathbb{R}^n$. The difference $|F(x)-F(x^*)|$ is called the excess error. Using Stochastic Gradient ...
1
vote
1answer
67 views

Newton step for ${\min}_{x \in \mathbb{R}^n} \ \sum_{i=1}^n -\ln(1 + \eta_i x_i) \ $ s.t. $A x \leq b$; $-x \leq 0$ to be used in primal-dual

I have a following problem on hand. P1: \begin{align} \text{minimize}_{x \in \mathbb{R}^n} \quad & \sum_{i=1}^n -\ln(1 + \eta_i x_i) \equiv -e^T \ln(e + \eta \odot x) \\ \text{subject to }\quad &...
1
vote
0answers
39 views

Handling singular matrices in gradient-descent optimization.

Right now I am coding up optimization for a 70 dimension nonlinear optimization, where the analytical gradient is unavailable. I have some non-linear constraints that maps the structural parameters ...
1
vote
0answers
43 views

Find minimizer of mean values

i'd like to know if there is an analytical method to solve the following optimisation problem : $\forall i=1,..,n$ find $\omega_i^{}$ and $\alpha_i^{}$ such that: $\dfrac{1}{n} \displaystyle \sum_{i=...
1
vote
0answers
60 views

Optimisation under constraint of Wasserstein distance

Let $\mathcal P_n = \{P \in \mathbb R^n_{\geq 0}: P^T \mathbb I = 1 \}$, where $\mathbb I = (1,...,1)^T \in \mathbb R^n$ and $f: \mathcal P_n \to \mathbb R$ a convex and differentiable function (or ...
1
vote
0answers
44 views

Use Conjugate Gradient to obtain the Eigenvalues

So I have been told that we can use CG to obtain an eigenvalue approximation to the true matrix. I am not sure how? (Connection to Lanczos) Furthermore I have been told that there is a deep ...
1
vote
1answer
41 views

Iterative algorithm for a simple linear optimization problem

Let $c_1,\dots,c_n$ be $n$ positive numbers and so be $a_1,\dots, a_n$. For some $r$ such that $1\leq r\leq n$, consider the optimization problem \begin{align} \max_{x_i\in\mathbb{R}}&~~\sum_{i=1}^...
1
vote
0answers
34 views

What methods can solve SOCP problems?

What methods can solve SOCP problems? I need at least few different. By: https://en.wikipedia.org/wiki/Second-order_cone_programming it seems that the problem "reduces" to simpler problems, but I'...
1
vote
0answers
59 views

Reference request. Rigorous numerical optimization

I am looking for texts on Numerical optimization that are closer to Analysis definition-theorem-proof style texts. EDIT: This is my first acquaintance with numerical optimization. My institution ...
1
vote
0answers
51 views

Nonlinear optimization with parametric constraint

Is there any way of reformulating the following problem so that it can be solved by means of e.g. Matlab's fmincon? $\min f(x_1,\dots,x_n)$ subject to $c(x_1,\dots,...
1
vote
0answers
20 views

On the idea of the penalty methods of constrained minimization

I'm studying constrained optimization. For instance, consider the EQ constrained optimization problem of the form \begin{align} min_{x \in X} && f(x)\\ s.t && h_i(x)=0 && i=...
1
vote
0answers
103 views

Projected Conjugate Gradient or BFGS for bound constrained optimization

We know how projected gradient descent works for bound constrained optimization (https://neos-guide.org/content/gradient-projection-methods). It is basically steepest descent with an additional ...
1
vote
0answers
22 views

The constrained optimization problem

I would like to find the minimum value $F(x)=x^{T}Ax$ and $\|x\|_{2}=1$, where $A$ is symmetric and positive-definite. I know that the minimum value is the smallest eigenvalue problem of the matrix $...
1
vote
0answers
90 views

Solving 2nd order ODE with variable coefficients

ODE: $$X''(t)+A(t)X'(t)+B(t)X(t)=F(t),$$ IC's: $$X(0)=U_1, $$ $$X'(0)=U_2$$ where $X(t)=[X_1 X_2...X_n]^T$ and $U_1, U_2,F(t)$ are $n\times1$ matrices, $A(t), B(t)$ are $n\times n$ matrices. ...
1
vote
0answers
22 views

An alternate for Householder QR linear equation solving for fixed-layout sparse matrix

This concerns sparse matrices where the sparsity pattern is known beforehand, and where the size is between 5 and up to 50, as the linear solver for a Newton Raphson non linear solver. For smaller ...
1
vote
0answers
40 views

Nonlinear optimization of a matrix with the costraint to be orthonormal

I'm trying to find the matrix x which minimize the following cost function : $J =||B_b -x*B_n||^2$ with the constraint that x has to be an orthonormal matrix. I'm trying to use MATLAB fmincon tool, ...
1
vote
0answers
11 views

What are the conditions for a coordinatewise optimum to be a local optimum

if $x*$ is a coordinatewise optimum of a function $f$, (i.e $x*$ is an optimum of $f$ in all directions), what are the conditions (on the function $f$ or $x*$) for $x*$ to be a local optimum of $f$.
1
vote
0answers
70 views

Using the Newton's method to find the global minimum of a 2D problem with a constraint

I am solving an optimization problem $$\min f(x_1,x_2)\\ \text{s.t.}~~~~ c(x_1,x_2)\leq 0$$ In my problem, there are two min and two max and I am looking for the global min. I know that with the ...
1
vote
0answers
29 views

Encouraging sparse output in optimization problem

I am trying to define an optimization problem: $$ \min_\theta \sum_{x\in X} L(x, \theta, f_\theta(x),\delta_\theta(x)) + \lambda_s S(\delta_\theta(x)) $$ where $X$ is the dataset, $\theta$ are the ...
1
vote
0answers
45 views

Schatten-1 norm as matrix constraint

suppose I have a tensor $x \in \mathbb{R}^{n \times 2 \times 3}$. I take the seminorm of $x$ given by taking the Schatten-1 Norm in every $2 \times 3$ slice and then the $\ell_1$-Norm of the resulting ...
1
vote
0answers
43 views

Data sets for linear inequality and equality constrained quadratic optimization.

I'm trying to find some test problems for an algorithm that is solving the problem below: $$ \begin{array}{ll} \text{minimize} & x^T Q x + px\\ \text{subject to} & Ax + b = 0\\ \text{ } &...
1
vote
0answers
39 views

How to convert a semidefinite program to an optimization problem that can be solved by the ellipsoid method?

Suppose we have the following set of linear inequalities in $x \in \mathbb R^n$ $$\begin{aligned} a_1^T x &\leq b_1\\ a_2^T x &\leq b_2\\ &\vdots\\ a_k^T x &\leq b_k\end{aligned}$$ ...
1
vote
0answers
32 views

Optimization with probabilistic variable distribution

Let $u \in \mathcal{R}^n$ be a vector of decision variables. Let $C(u)$ be a function $\mathcal{R}^n \to \mathcal{R}$ that is a measure of performance of the variables. Further, it is required that ...
1
vote
0answers
29 views

Efficient numerical optimization of an “almost separable” function

I have come across an optimization problem with the following objective function: $$f(x_0,y_0,z_0,x_1,y_1,z_1,...,x_N,y_N,z_N) = \sum_{i=0}^N f_i(x_i,y_i,z_i, \alpha(x_{i+1}-x_i) + \beta(y_{i+1}-y_i))...
1
vote
0answers
26 views

Advantages of the LM algorithm over Quasi-Newton

I have an optimization problem that I'm solving using matlab, I didn't pay much attention to what kind of problem was except that it wasn't linear. So I solved it using a ...
1
vote
0answers
30 views

Algorithm optimization

Let x $\in {R^{n\times1}}$ so that $x=(x_1,x_2,\dots,x_n)^T$ and $\textbf{diffmax} $ represents the absolute difference between two consecutive numbers , $\textbf{diffmax>=max\{$|x_i - x_{i+1}|,...
1
vote
0answers
170 views

Box constrained optimization - BFGS

I have written my own code to implement BFGS method for unconstrained problem(FORTRAN). But now I want to convert the same code for solving box constrained optimization problems. How can I go about to ...
1
vote
0answers
86 views

Numerical Optimization - tracking Floating point arithmetic round off error

Let's say I am working on to find the minimum of a function using quasi-newton method. As in every iteration, there will be many function evaluations and as we are dealing with floating point ...
1
vote
0answers
49 views

Is there a variational problem that can provide the following class of variational derivative?

Suppose I have the variational problem $$ E(y) = \frac{1}{2}\int_{a}^{b} y^2 + \alpha y'^2dx $$ Variational derivative will provide $$ \frac{\delta E}{ \delta y} = y -\alpha y'', $$ Is there a ...
1
vote
0answers
24 views

Finding solution of matrix equation system $XA_iY=B_i$ with respect to $X$ and $Y$

We have matrices $A_i$, $B_i \in\mathbb R^{4\times4}, i \in \{1, \ldots, K\}$ and we'd like to find $X$, $Y \in R^{4\times4}$ such that $X A_i Y = B_i$ Is there any (maybe numeric) way to solve it ...
1
vote
0answers
35 views

How to solve the bilevel optimization problem stated here?

How do you usually describe the following problem and how to solve it ? \begin{equation*} \begin{aligned} & \underset{x,y}{\min} & & f(x,y) + z, \\ & \text{s.t.} & & z = \...
1
vote
0answers
54 views

Gradient descent versus finding where the gradient vanishes via solving systems of equations

I started learning machine learning and got stuck at the following questions: Why do we need to iterate the gradient descent algorithm? Why don't we equate the gradient to zero and find all local ...
1
vote
1answer
108 views

How to maximize the “k-largest” functions?

I want to solve the following optimization problem: $$\max_{x} ~sumk(A\vec{x})$$ $$s.t ~~~ x \geq 0$$ $$~~~~~~~ \sum_i x_i =1 \quad\forall i=1,...,N$$ in which, $A$ and $x$ are matrix and vector ...
1
vote
0answers
59 views

How to improve the numerical stability of the inverse rank-one Cholesky update?

I am trying to use the inverse Cholesky update from the page 10 of the Efficient covariance matrix update for variable metric evolution strategies paper as a part of the optimization step in a neural ...
1
vote
1answer
113 views

When to use which “closed” Newton Cotes rule?

Given a set of datapoints, I was thinking about when to use which (closed) Newton-Cotes formula? I developed a decision tree which would go like this: Are the given datapoints equally spaced? ...
1
vote
0answers
63 views

Solving mathematical optimization problems with nonnegative, non-analytical objective functions

I am studying a kind of nonlinear mathematical optimization problems where the objective functions are guaranteed to be nonnegative. In addition, the objective function is non-analytical; in fact, it ...
1
vote
1answer
83 views

Adapting gradient descent for numerical functional optimization?

Let's say we have a functional $J[f] = \int_a^b L(x, f, f') dx$ that we are trying to minimize. I'm trying to think what is the best way to do this numerically, with something like gradient descent. ...
1
vote
0answers
27 views

mating proposal for genetic algorithms

I am using a genetic algorithm to solve an optimization problem. However, I want a mating process that enables to create new solutions that are outside the range of current solutions. In the book: ...
1
vote
0answers
110 views

Life cycle model in Matlab

I am currently working on a life cycle model about saving and investing for pension funds. However, I have some problems solving the model. The model runs for $T$ years over the life cycle. At every ...
1
vote
0answers
31 views

good books that explained optimization using SA,Genetics and … algorithm

I don't know this is right place for asking this but I had no Other options In Operation Research Course My professor give us some lecture note about nonlinear optimization and some algorithm for ...