Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [numerical-optimization]

Numerical methods for continuous optimization.

1
vote
1answer
24 views

Numerical solution of non-linear first order partial differential equation (HJB)

I am trying to solve a simple optimal control problem using the Hamilton-Jacobi-Bellman equation, numerically in Python. This is proving to be rather difficult as I end up having to solve the ...
1
vote
1answer
45 views

Find the maximum value without computing matrix

I have a optimization problem, which is to find a certain $h^*$ that:$$h^* = argmax(h'\alpha-\frac{\kappa}{2}h'\Sigma h)$$ where $\alpha$ is a $(n \times 1)$ vector and $\Sigma$ is a $(n\times n)$ ...
0
votes
0answers
19 views

Is dynamic programming suitable for a specific optimization problem?

Let $c,\,\mathcal{P}_0,\,\mathcal{P}_1,\,\mathcal{P}_2,\ldots$ be a sequence of positive real numbers. Let $N\in\{1,\,2,\,3,\ldots\}$ and let $t\in\{0,\,1,\,2,\ldots\}$, with $N$ and $t$ fixed. ...
0
votes
1answer
17 views

Optimization problem: Computing the gradient [closed]

I need help with the following exercise: Solve $\min_{x\in\mathbb{R}^d} f(x)$, where $f:\mathbb{R}^d\to\mathbb{R}$. We define the inner product $(v,w)_A:= v^TAw$, induced by a positive definite and ...
0
votes
0answers
13 views

Tychonoff Regularization by calling an Optimization Routine

Question : Set $ X = [−1,1]$ let $u_c(x)=sin(\pi x) $ be a clean signal. Add noise $n(x)$ which is mean zero with variance $σ^2=0.1^2$ and let $u_n=u+n$. Let, $ 0 = x_1,......,x_n = 1$ be an equally ...
1
vote
0answers
11 views

An alternate for Householder QR linear equation solving for fixed-layout sparse matrix

This concerns sparse matrices where the sparsity pattern is known beforehand, and where the size is between 5 and up to 50, as the linear solver for a Newton Raphson non linear solver. For smaller ...
0
votes
0answers
23 views

Roots of a real exponential sum

Suppose I had some exponential sum $\ f(x)\ $ of the form: $$f(x) = \sum_{i=1}^{N} \left( c_i \ e^{a_i x} \right)$$ where: $$c_i, a_i \in R$$ $$a_i \leq 0$$ Is there a quick way to find the roots, $\ \...
0
votes
0answers
11 views

Equality and inequality constraints in multi-objective optimisation?

The general form of the multi-objective optimisation as the following: ...
1
vote
0answers
17 views

Nonlinear optimization of a matrix with the costraint to be orthonormal

I'm trying to find the matrix x which minimize the following cost function : $J =||B_b -x*B_n||^2$ with the constraint that x has to be an orthonormal matrix. I'm trying to use MATLAB fmincon tool, ...
0
votes
0answers
33 views

Numerical zeros of a nonnegative function?

Suppose we have a function $f:\mathbb{R}^{n}\rightarrow \mathbb{R}$ that satisfies $f\left(x\right)\geq 0$, for every $x\in\mathbb{R}^{n}$. What is the good numerical way of finding points $x_{0}\in\...
0
votes
1answer
18 views

Newton-Raphson on strictly convex function

Maybe someone can give me a hint here: question 1: Given a sequence {$x_n$} which is the Newton-Raphson sequence on some $f(x)$ s.t. $f(x)$ is strictly convex and $f'>0$. Let $\alpha$ be the ...
0
votes
0answers
14 views

(Worst and average case) Arithmetic and time complexity of a quadratic programming problem by interior point method

My understanding of complexity measures are at basic level. So, please excuse me for asking basic question, if it sounds like. I am trying to understand the arithmetic and time complexity of a ...
0
votes
2answers
19 views

The meaning of active and non-active constraints

Consider the constrained minimization problem min $f(x), x \in \mathbb{R^n}$ s.t $h_i(x)=0, i=1,2,...m$ $g_i(x) \leq 0 , i=1,2,..k$ Now the author states: " For a feasible solution $x$, some of ...
0
votes
0answers
23 views

A question about the equality of the solutions to primal and dual problems

I am quite new in dynamic programming and especially the duality theory, so there is a question I would like to inquiry and confirm. Given a primal dynamic optimization problem (P) in the infinite ...
0
votes
0answers
6 views

What are the conditions for a coordinatewise optimum to be a local optimum

if $x*$ is a coordinatewise optimum of a function $f$, (i.e $x*$ is an optimum of $f$ in all directions), what are the conditions (on the function $f$ or $x*$) for $x*$ to be a local optimum of $f$.
1
vote
0answers
25 views

Using the Newton's method to find the global minimum of a 2D problem with a constraint

I am solving an optimization problem $$\min f(x_1,x_2)\\ \text{s.t.}~~~~ c(x_1,x_2)\leq 0$$ In my problem, there are two min and two max and I am looking for the global min. I know that with the ...
0
votes
0answers
32 views

Defining a Jacobian Matrix

reaching out my post. I have a trouble of defining a Jacobian matrix for my problem. Basically, I have 4 differential equations to be solved. $\dot x_1(t)=x_2(t),$ $\dot x_2(t)=p_2(t)−\sqrt 2 x_1(t)...
0
votes
0answers
18 views

Standard name for the computation of Lagrange multipliers iteratively by fixing other multipliers?

Dear Optimization Experts, Background: I have a convex optimization problem on hand that can be shown in general form as given below \begin{equation} \begin{aligned} & \underset{x \in \mathbb{R}...
0
votes
0answers
30 views

Production time optimization model

I have a problem where I have 2 different types of raw-material. These materials can be processed in two different production-lines which take different amount of time, depending on raw material. ...
0
votes
0answers
15 views

Hyperparameter optimization

How to choose hyper-parameters for optimisation methods in practice? I know that there are hyper-parameter optimisation techniques such as gradient-based or bayesian methods, but for instance it is ...
1
vote
0answers
20 views

Numerical Integration Schemes for a Hemispherical Region

I am struggling with a problem that requires me to numerical integrate a function within a hemisphere. While the function is smooth, it is pretty nasty, extremely oscillatory and I have to calculate ...
0
votes
3answers
21 views

Determining coefficients of a parametrization of an epicycloid given a predefined arc length.

I am trying to determine the coefficient q in the parametrization of a epicycloid which gives me the arc length of 4.25. The parametrization can be glimpsed in my attempt of a solution in the ...
0
votes
0answers
19 views

What are the directions of research in Numerical Optimization?

I have just begun reading in the field of Numerical Optimization. Are people trying to invent new Algorithms? or proving the convergence of Heuristic Algorithms? and what else? What are the tools a ...
0
votes
0answers
11 views

Solving multiobjective problem with matlb

I would like to solve a multiobjective problem with matlab with NSGA II procedure. The problem is a maximization/minimizationf objective functions. Can someone porvide me this code, and explain how to ...
0
votes
1answer
36 views

Maximise $z = \frac{y}{2x+2y}+\frac{50-y}{200-2x-2y}$ given that $x+y$ is non zero and $x+y<100$. Also, $x\leq50$ and $y\leq50$ and non-negative.

Z is actually a probability function. I am finding where the probability is maximized. But I could find no way how to maximize this function. Original question is as follows: Mr A wants to join a ...
0
votes
1answer
47 views

Numerical integration in Finite Element Method (and implementation in Matlab)?

i'm trying to solve the p-Laplace Equation: \begin{align} \begin{cases} \text{div} (\sigma |\nabla u|^{p-2} \nabla u) = f &\quad \text{in } \Omega\\ u = g &\quad \text{in } \partial\Omega \...
1
vote
0answers
15 views

Encouraging sparse output in optimization problem

I am trying to define an optimization problem: $$ \min_\theta \sum_{x\in X} L(x, \theta, f_\theta(x),\delta_\theta(x)) + \lambda_s S(\delta_\theta(x)) $$ where $X$ is the dataset, $\theta$ are the ...
0
votes
0answers
7 views

Numerically determining if a critical point is a saddle point in the presence of inequality constraints

I have a constrained optimization problem $$\min_{\mathbf{x}} f(\mathbf{x}) \quad \mathrm{s.t.}\quad g(\mathbf{x}) = \mathbf{0}, h(\mathbf{x}) \geq \mathbf{0}$$ and need to probe if a critical point $\...
0
votes
0answers
56 views

Minimizing a Gaussian Mixture

I'm trying to find the local minima, if they exist, of, $$G(r) = \sum_{n=1}^N \beta_n e^{{-(r-r_n)}^2}$$ Such that $r$, $r_n$, and $\beta_n \in \Bbb R^+$ are positive scalars. Edit: The $r_n$ are ...
0
votes
0answers
23 views

Bad convex functions

Why are functions of the form $ \ f_{m} (x) = \eta_{p+1}(A_m x) - x_1$ bad, where $\eta_{p+1} = \frac{1}{p+1}\sum_{i=1}^{n}(x_i)^{p+1}$, $A_m = \begin{pmatrix}U_m & 0 \\ 0 &I_{n-m} \end{...
0
votes
0answers
14 views

Schatten-1 norm as matrix constraint

suppose I have a tensor $x \in \mathbb{R}^{n \times 2 \times 3}$. I take the seminorm of $x$ given by taking the Schatten-1 Norm in every $2 \times 3$ slice and then the $\ell_1$-Norm of the resulting ...
2
votes
0answers
31 views

Self-study - Numerical Optimization

Is there any available online course on Numerical Optimization other than NPTEL courses? I'm also looking for lecture notes or textbooks that are suitable for self-study with lots of examples and ...
0
votes
0answers
21 views

Galerkin Approximation

Let $u_N$ be the Galerkin-Approximation of u and $\dim(V_N)=N$. Suppose that the error $\Vert u_N - u \Vert$ can be developed in a power series $\sum_{i=0}^{\infty}C_i(N^{\alpha})^i$ with $C_i,\alpha\...
0
votes
0answers
17 views

Global convergence of the BFGS method

Please help me prove that $$\det(B_{k+1})=\det(B_k)\frac{y_k^Ts_k}{s_k^TB_ks_k}$$ This is from the global convergence of the BFGS method (Quasi-Newton methods), where $B_k$ is the approximate ...
0
votes
0answers
26 views

Rate of convergence of the steepest descent method on a general linear function

Can someone help me with the proof of this theorem (it's theorem 3.4 from the Nocedal and Wright book - Chapter 3 - Line search methods) regarding the convergence rate of the steepest descent method ...
0
votes
0answers
17 views

Is a hypergeometric sum the minimum of a “potential” function?

I'm wondering if values of a generalized hypergeometric function can be written as solutions to an optimization problem, like this: $$_q F_p (a_1, \dots, a_p;b_1,\dots,b_q;x)=\min_{t}\psi(a_1, \dots, ...
0
votes
0answers
24 views

Nonlinear optimization with complex residual and jacobian

I am trying to minimize the following function $\chi^2=(f(t_i;\vec{p})-y_i)^{H}\text{Cov}^{-1}_{ij}(f(t_j;\vec{p})-y_j)$ where $A^H$ is the hermitian conjugate of A, $f(t;\vec{p})-y$ is a complex ...
1
vote
1answer
35 views

Continuity of supremum of polynomial

Prove: Let $A \subseteq \mathbb{R}$ be a compact set. Prove that the function $f \colon\mathbb{R^{n+1}} \to \mathbb{R}$ $\qquad f(x_0,..., x_n) = \sup_{x\in A} \prod_{j=0}^{n} (x-x_j)$ is continuous....
2
votes
1answer
108 views

Second-Order Taylor Series Terms In Gradient Descent

My machine learning textbook states the following when discussing second-order Taylor series approximations in the context of Gradient descent: The (directional) second derivative tells us how well ...
0
votes
1answer
26 views

determine matrix and vector to fit regularized normal equation

I hope the title is not too unclear. I am given a Matrix $$A\in~\mathbb{R}^{K\times~N},~b\in~\mathbb{R}^{K}$$ and instead of solving the normal equation $min_{x\in~\mathbb{R}^N}|Ax-b|^2_2,~$ an $\...
0
votes
0answers
53 views

Is it impossible that gradient descent converges to strict saddle point?

Suppose $f(x), x \in X \subset R^n$, is a smooth function (or real analytic function) which has only one stationary point $x^* \in X$, ($\nabla f(x^*)=0$), and $x^*$ is a strict saddle point which ...
0
votes
0answers
108 views

Proof of sub-multiplicative property in Frobenius norm for $n\times n$ matrices $||AB||_{F}\leq||A||_{F}||B||_{F}$

I wanted to share with you this little demonstration I got making some homework, it seems to be beautiful for me, also it would be nice to recibe some feedback. I would also like to clear out that it ...
0
votes
0answers
7 views

Branch of optimization that combines stochastic search to numerical optimization

Is there any branch of optimization that combines techniques of numerical optimization with stochastic search? Like we have gradient descent, but I saw in some classes (like machine learning) the ...
0
votes
2answers
30 views

stopping criteria for mathematical optimisation: objective function target, rather than convergence

Researching stopping criteria for mathematical-optimisation algorithms, any libraries I look at (e.g. matlab, apache commons math) only have iteration limits and convergence criteria (e.g. convergence ...
0
votes
0answers
17 views

trust region optimization conjugate gradient steihaug subproblem

I'm trying to solve by hand the trust region optimization conjugate gradient - Steihaug method from the book Numerical Optimization by Nocedal and Wright - Algorithm 7.2 as shown below. I'm struggling ...
0
votes
0answers
47 views

Finding the Geometric Median of N points in 2D Euclidean space when its X co-ordinate is given.

I have a given cluster of N points in 2-space. The goal is to find the geometric median of this cluster, and the x co-ordinate of the geometric median is known. What is the most computationally ...
1
vote
0answers
39 views

Data sets for linear inequality and equality constrained quadratic optimization.

I'm trying to find some test problems for an algorithm that is solving the problem below: $$ \begin{array}{ll} \text{minimize} & x^T Q x + px\\ \text{subject to} & Ax + b = 0\\ \text{ } &...
0
votes
1answer
24 views

- Optimization - Standard Grid Search

I'm struck into an portfolio opt. problem and the paper I'm replicating (or, better, trying to) is using a "Standard Grid Search". Since I never encountered it before, I would like to ask you about: ...
0
votes
0answers
55 views

nocedal and wright singular values bounded away from zero

In the Nocedal and Wright Numerical optimization second edition book, pages 255-256, they state that the Jacobians "$J(x)$ have their singular values uniformly bounded away from zero in the region of ...
2
votes
0answers
23 views

Methods for numerically solving field shapes for multiple permanent magnets.

I am trying to optimize an engineering setup which involves an arrangement of permanent neodymium magnets in 3d space. I am planning on using Houdini to solve the optimization as a simulation, which ...