Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

Filter by
Sorted by
Tagged with
0
votes
0answers
37 views

Production time optimization model

I have a problem where I have 2 different types of raw-material. These materials can be processed in two different production-lines which take different amount of time, depending on raw material. ...
0
votes
0answers
28 views

Hyperparameter optimization

How to choose hyper-parameters for optimisation methods in practice? I know that there are hyper-parameter optimisation techniques such as gradient-based or bayesian methods, but for instance it is ...
2
votes
0answers
23 views

Numerical Integration Schemes for a Hemispherical Region

I am struggling with a problem that requires me to numerical integrate a function within a hemisphere. While the function is smooth, it is pretty nasty, extremely oscillatory and I have to calculate ...
0
votes
1answer
98 views

Numerical integration in Finite Element Method (and implementation in Matlab)?

i'm trying to solve the p-Laplace Equation: \begin{align} \begin{cases} \text{div} (\sigma |\nabla u|^{p-2} \nabla u) = f &\quad \text{in } \Omega\\ u = g &\quad \text{in } \partial\Omega \...
0
votes
1answer
102 views

Maximise $z = \frac{y}{2x+2y}+\frac{50-y}{200-2x-2y}$ given that $x+y$ is non zero and $x+y<100$. Also, $x\leq50$ and $y\leq50$ and non-negative.

Z is actually a probability function. I am finding where the probability is maximized. But I could find no way how to maximize this function. Original question is as follows: Mr A wants to join a ...
10
votes
3answers
17k views

Optimal step size in gradient descent

Suppose a differentiable, convex function $F(x)$ exists. Then $b = a - \gamma\nabla F(a)$ implies that $F(b) \leq F(a)$ given $\gamma$ is chosen properly. The goal is to find the optimal $\gamma$ at ...
0
votes
0answers
23 views

What are the directions of research in Numerical Optimization?

I have just begun reading in the field of Numerical Optimization. Are people trying to invent new Algorithms? or proving the convergence of Heuristic Algorithms? and what else? What are the tools a ...
0
votes
0answers
12 views

Solving multiobjective problem with matlb

I would like to solve a multiobjective problem with matlab with NSGA II procedure. The problem is a maximization/minimizationf objective functions. Can someone porvide me this code, and explain how to ...
1
vote
0answers
27 views

Encouraging sparse output in optimization problem

I am trying to define an optimization problem: $$ \min_\theta \sum_{x\in X} L(x, \theta, f_\theta(x),\delta_\theta(x)) + \lambda_s S(\delta_\theta(x)) $$ where $X$ is the dataset, $\theta$ are the ...
0
votes
0answers
11 views

Numerically determining if a critical point is a saddle point in the presence of inequality constraints

I have a constrained optimization problem $$\min_{\mathbf{x}} f(\mathbf{x}) \quad \mathrm{s.t.}\quad g(\mathbf{x}) = \mathbf{0}, h(\mathbf{x}) \geq \mathbf{0}$$ and need to probe if a critical point $\...
0
votes
0answers
60 views

Minimizing a Gaussian Mixture

I'm trying to find the local minima, if they exist, of, $$G(r) = \sum_{n=1}^N \beta_n e^{{-(r-r_n)}^2}$$ Such that $r$, $r_n$, and $\beta_n \in \Bbb R^+$ are positive scalars. Edit: The $r_n$ are ...
1
vote
0answers
41 views

Schatten-1 norm as matrix constraint

suppose I have a tensor $x \in \mathbb{R}^{n \times 2 \times 3}$. I take the seminorm of $x$ given by taking the Schatten-1 Norm in every $2 \times 3$ slice and then the $\ell_1$-Norm of the resulting ...
2
votes
0answers
40 views

Self-study - Numerical Optimization

Is there any available online course on Numerical Optimization other than NPTEL courses? I'm also looking for lecture notes or textbooks that are suitable for self-study with lots of examples and ...
0
votes
0answers
29 views

Galerkin Approximation

Let $u_N$ be the Galerkin-Approximation of u and $\dim(V_N)=N$. Suppose that the error $\Vert u_N - u \Vert$ can be developed in a power series $\sum_{i=0}^{\infty}C_i(N^{\alpha})^i$ with $C_i,\alpha\...
0
votes
0answers
48 views

Global convergence of the BFGS method

Please help me prove that $$\det(B_{k+1})=\det(B_k)\frac{y_k^Ts_k}{s_k^TB_ks_k}$$ This is from the global convergence of the BFGS method (Quasi-Newton methods), where $B_k$ is the approximate ...
0
votes
0answers
65 views

Rate of convergence of the steepest descent method on a general linear function

Can someone help me with the proof of this theorem (it's theorem 3.4 from the Nocedal and Wright book - Chapter 3 - Line search methods) regarding the convergence rate of the steepest descent method ...
0
votes
1answer
31 views

determine matrix and vector to fit regularized normal equation

I hope the title is not too unclear. I am given a Matrix $$A\in~\mathbb{R}^{K\times~N},~b\in~\mathbb{R}^{K}$$ and instead of solving the normal equation $min_{x\in~\mathbb{R}^N}|Ax-b|^2_2,~$ an $\...
0
votes
0answers
23 views

Is a hypergeometric sum the minimum of a “potential” function?

I'm wondering if values of a generalized hypergeometric function can be written as solutions to an optimization problem, like this: $$_q F_p (a_1, \dots, a_p;b_1,\dots,b_q;x)=\min_{t}\psi(a_1, \dots, ...
0
votes
0answers
52 views

Nonlinear optimization with complex residual and jacobian

I am trying to minimize the following function $\chi^2=(f(t_i;\vec{p})-y_i)^{H}\text{Cov}^{-1}_{ij}(f(t_j;\vec{p})-y_j)$ where $A^H$ is the hermitian conjugate of A, $f(t;\vec{p})-y$ is a complex ...
2
votes
1answer
293 views

Second-Order Taylor Series Terms In Gradient Descent

My machine learning textbook states the following when discussing second-order Taylor series approximations in the context of Gradient descent: The (directional) second derivative tells us how well ...
1
vote
1answer
77 views

Continuity of supremum of polynomial

Prove: Let $A \subseteq \mathbb{R}$ be a compact set. Prove that the function $f \colon\mathbb{R^{n+1}} \to \mathbb{R}$ $\qquad f(x_0,..., x_n) = \sup_{x\in A} \prod_{j=0}^{n} (x-x_j)$ is continuous....
0
votes
0answers
76 views

Is it impossible that gradient descent converges to strict saddle point?

Suppose $f(x), x \in X \subset R^n$, is a smooth function (or real analytic function) which has only one stationary point $x^* \in X$, ($\nabla f(x^*)=0$), and $x^*$ is a strict saddle point which ...
1
vote
0answers
681 views

How does one rigorously prove that gradient descent indeed decreases the function in question locally i.e. show $f(x^{(t+1)}) \leq f(x^{(t)})$?

How does one prove that gradient descent indeed decreases the function in question locally? In other words if we take a step in the negative of the gradient as in: $$ x^{(t+1)} = x^{(t)} - \eta \...
0
votes
0answers
257 views

Proof of sub-multiplicative property in Frobenius norm for $n\times n$ matrices $||AB||_{F}\leq||A||_{F}||B||_{F}$

I wanted to share with you this little demonstration I got making some homework, it seems to be beautiful for me, also it would be nice to recibe some feedback. I would also like to clear out that it ...
0
votes
0answers
12 views

Branch of optimization that combines stochastic search to numerical optimization

Is there any branch of optimization that combines techniques of numerical optimization with stochastic search? Like we have gradient descent, but I saw in some classes (like machine learning) the ...
0
votes
2answers
48 views

stopping criteria for mathematical optimisation: objective function target, rather than convergence

Researching stopping criteria for mathematical-optimisation algorithms, any libraries I look at (e.g. matlab, apache commons math) only have iteration limits and convergence criteria (e.g. convergence ...
0
votes
0answers
58 views

trust region optimization conjugate gradient steihaug subproblem

I'm trying to solve by hand the trust region optimization conjugate gradient - Steihaug method from the book Numerical Optimization by Nocedal and Wright - Algorithm 7.2 as shown below. I'm struggling ...
0
votes
0answers
109 views

Finding the Geometric Median of N points in 2D Euclidean space when its X co-ordinate is given.

I have a given cluster of N points in 2-space. The goal is to find the geometric median of this cluster, and the x co-ordinate of the geometric median is known. What is the most computationally ...
4
votes
1answer
408 views

Is there an intuition for cyclic monotonicity?

Cyclic monotonicity says that if we have a correspondence, $x(w)$, the $x$ is cyclically monotone in $w$ if for a finite sequence $w_1,\cdots w_k$ and $x^*(w_i)\in x(w_i)$, we have $\sum_{i=1}^k (w_i-...
1
vote
0answers
42 views

Data sets for linear inequality and equality constrained quadratic optimization.

I'm trying to find some test problems for an algorithm that is solving the problem below: $$ \begin{array}{ll} \text{minimize} & x^T Q x + px\\ \text{subject to} & Ax + b = 0\\ \text{ } &...
0
votes
1answer
56 views

- Optimization - Standard Grid Search

I'm struck into an portfolio opt. problem and the paper I'm replicating (or, better, trying to) is using a "Standard Grid Search". Since I never encountered it before, I would like to ask you about: ...
0
votes
0answers
64 views

nocedal and wright singular values bounded away from zero

In the Nocedal and Wright Numerical optimization second edition book, pages 255-256, they state that the Jacobians "$J(x)$ have their singular values uniformly bounded away from zero in the region of ...
6
votes
2answers
2k views

Intuition for gradient descent with Nesterov momentum

A clear article on Nesterov’s Accelerated Gradient Descent (S. Bubeck, April 2013) says The intuition behind the algorithm is quite difficult to grasp, and unfortunately the analysis will not be ...
2
votes
0answers
25 views

Methods for numerically solving field shapes for multiple permanent magnets.

I am trying to optimize an engineering setup which involves an arrangement of permanent neodymium magnets in 3d space. I am planning on using Houdini to solve the optimization as a simulation, which ...
1
vote
0answers
32 views

Optimization with probabilistic variable distribution

Let $u \in \mathcal{R}^n$ be a vector of decision variables. Let $C(u)$ be a function $\mathcal{R}^n \to \mathcal{R}$ that is a measure of performance of the variables. Further, it is required that ...
0
votes
1answer
55 views

Convergence of Laczos method

In general it is a well established result that the Lanczos method, as a Krylov subspace method, can converge quadratically faster than the simple power method. In particular, the proof (using the ...
2
votes
1answer
100 views

Maximize the modulus of a sum of complex exponentials on a given interval

I am trying to find $$\hat{t}=\underset{t \in [a,b]}{\text{argmax}}\,\left|\sum_{j=1}^n e^{i x_j t}\right|$$ where $t$ is a real number, $[a,b]$ is a given interval, $i$ is $\sqrt{-1}$, and $x_j$ are ...
1
vote
0answers
29 views

Efficient numerical optimization of an “almost separable” function

I have come across an optimization problem with the following objective function: $$f(x_0,y_0,z_0,x_1,y_1,z_1,...,x_N,y_N,z_N) = \sum_{i=0}^N f_i(x_i,y_i,z_i, \alpha(x_{i+1}-x_i) + \beta(y_{i+1}-y_i))...
1
vote
1answer
276 views

Linear Programming, Optimal Solutions

I posted the whole question to give some context, but my problem lies with (iv). I think you're meant to use a formula for the generalization of the optimal solution, but I'm not really sure what this ...
0
votes
0answers
12 views

Amortized Approximate Map Recovery

I want to implement Sparse Extended Information Filter Slam.I studied it from Probabilistic Robotics by Dr. Sebestian Thrun. I have some numerical doubt in chapter 12 page 321(Amortized Approximate ...
0
votes
0answers
121 views

ADMM difficulty in finding active inequality constraints.

I have implemented a convex optimization algorithm based on the ADMM approach for quadratic programming of the form below: $$ \begin{array}{ll} \text{minimize} & x^T Q x + px\\ \text{subject to} &...
1
vote
0answers
30 views

Algorithm optimization

Let x $\in {R^{n\times1}}$ so that $x=(x_1,x_2,\dots,x_n)^T$ and $\textbf{diffmax} $ represents the absolute difference between two consecutive numbers , $\textbf{diffmax>=max\{$|x_i - x_{i+1}|,...
1
vote
0answers
23 views

Advantages of the LM algorithm over Quasi-Newton

I have an optimization problem that I'm solving using matlab, I didn't pay much attention to what kind of problem was except that it wasn't linear. So I solved it using a ...
0
votes
0answers
64 views

Can a numerical optimization algorithm get stuck into local maxima?

I've designed a cost function of the form $$ c(x) = \sum_j f_j(x) + \lambda \sum_j g_j(x) $$ which I'm trying to minimize (it's more specifically a non linear least square problem). When I run my ...
6
votes
3answers
137 views

Understanding Newton method when optimizing cost function depending on a triangle.

I have the following function defined from $\mathbb{R}^9 \to \mathbb{R}$ \begin{equation} f(x) = f(x_1,x_2,x_3) = \frac{1}{2}(\left\langle n,e_3 \right\rangle - \lVert n \rVert)^2 \end{equation} ...
0
votes
1answer
203 views

Gradient descent on non-linear function with linear constraints

Here is an optimization problem I'm trying to solve: Objective function to be minimized: $$ f(x) = -\sum_{i=1}^{n}(x_{i}+a_{i})\bigg[1-\exp\bigg(-\frac{x_ib_i}{x_i+a_i}\bigg)\bigg] $$ where the ...
25
votes
1answer
14k views

What is the difference between projected gradient descent and ordinary gradient descent?

I just read about projected gradient descent but I did not see the intuition to use Projected one instead of normal gradient descent. Would you tell me the reason and preferable situations of ...
0
votes
0answers
46 views

If the relative value of the most negative eigenvalue is small, can we view the quadratic program as convex?

Consider symmetric matrix $A \in \mathbb{R}^{n \times n}$ with $n-1$ negative eigenvalues and a positive one such that $\lambda_1 > \lambda_2> \dots > \lambda_n$, where $\lambda_n$ is the ...
0
votes
0answers
36 views

Quadratic Matrix programming

I would like to find a numerical solution to the following quadratic matrix programming problem. Square matrix A has thousands of rows, so I need a algorithm which works fast. I looked into Python but ...
0
votes
0answers
103 views

The selection coordinates method of coordinates in the coordinate descent/ascent algorithm?

The selection coordinates method of coordinates in the coordinate descent/ascent algorithm? What I know: Cyclic select coordinates; Randomly select coordinates (with replace); Randomly select ...