Questions tagged [numerical-optimization]
Numerical methods for continuous optimization.
1
vote
1answer
24 views
Numerical solution of non-linear first order partial differential equation (HJB)
I am trying to solve a simple optimal control problem using the Hamilton-Jacobi-Bellman equation, numerically in Python. This is proving to be rather difficult as I end up having to solve the ...
1
vote
1answer
45 views
Find the maximum value without computing matrix
I have a optimization problem, which is to find a certain $h^*$ that:$$h^* = argmax(h'\alpha-\frac{\kappa}{2}h'\Sigma h)$$
where $\alpha$ is a $(n \times 1)$ vector and $\Sigma$ is a $(n\times n)$ ...
0
votes
0answers
19 views
Is dynamic programming suitable for a specific optimization problem?
Let $c,\,\mathcal{P}_0,\,\mathcal{P}_1,\,\mathcal{P}_2,\ldots$ be a sequence of positive real numbers.
Let $N\in\{1,\,2,\,3,\ldots\}$ and let $t\in\{0,\,1,\,2,\ldots\}$, with $N$ and $t$ fixed.
...
0
votes
1answer
17 views
Optimization problem: Computing the gradient [closed]
I need help with the following exercise:
Solve $\min_{x\in\mathbb{R}^d} f(x)$, where $f:\mathbb{R}^d\to\mathbb{R}$. We define the inner product $(v,w)_A:= v^TAw$, induced by a positive definite and ...
0
votes
0answers
13 views
Tychonoff Regularization by calling an Optimization Routine
Question : Set $ X = [−1,1]$ let $u_c(x)=sin(\pi x) $ be a clean signal. Add noise $n(x)$ which is mean zero with variance $σ^2=0.1^2$ and let $u_n=u+n$. Let, $ 0 = x_1,......,x_n = 1$ be an equally ...
1
vote
0answers
11 views
An alternate for Householder QR linear equation solving for fixed-layout sparse matrix
This concerns sparse matrices where the sparsity pattern is known beforehand, and where the size is between 5 and up to 50, as the linear solver for a Newton Raphson non linear solver.
For smaller ...
0
votes
0answers
23 views
Roots of a real exponential sum
Suppose I had some exponential sum $\ f(x)\ $ of the form:
$$f(x) = \sum_{i=1}^{N} \left( c_i \ e^{a_i x} \right)$$
where:
$$c_i, a_i \in R$$
$$a_i \leq 0$$
Is there a quick way to find the roots, $\ \...
0
votes
0answers
11 views
Equality and inequality constraints in multi-objective optimisation?
The general form of the multi-objective optimisation as the following:
...
1
vote
0answers
17 views
Nonlinear optimization of a matrix with the costraint to be orthonormal
I'm trying to find the matrix x which minimize the following cost function :
$J =||B_b -x*B_n||^2$
with the constraint that x has to be an orthonormal matrix.
I'm trying to use MATLAB fmincon tool, ...
0
votes
0answers
33 views
Numerical zeros of a nonnegative function?
Suppose we have a function $f:\mathbb{R}^{n}\rightarrow \mathbb{R}$ that satisfies $f\left(x\right)\geq 0$, for every $x\in\mathbb{R}^{n}$. What is the good numerical way of finding points $x_{0}\in\...
0
votes
1answer
18 views
Newton-Raphson on strictly convex function
Maybe someone can give me a hint here:
question 1: Given a sequence {$x_n$} which is the Newton-Raphson sequence on some $f(x)$ s.t. $f(x)$ is strictly convex and $f'>0$.
Let $\alpha$ be the ...
0
votes
0answers
14 views
(Worst and average case) Arithmetic and time complexity of a quadratic programming problem by interior point method
My understanding of complexity measures are at basic level. So, please excuse me for asking basic question, if it sounds like.
I am trying to understand the arithmetic and time complexity of a ...
0
votes
2answers
19 views
The meaning of active and non-active constraints
Consider the constrained minimization problem
min $f(x), x \in \mathbb{R^n}$
s.t $h_i(x)=0, i=1,2,...m$
$g_i(x) \leq 0 , i=1,2,..k$
Now the author states:
" For a feasible solution $x$, some of ...
0
votes
0answers
23 views
A question about the equality of the solutions to primal and dual problems
I am quite new in dynamic programming and especially the duality theory, so there is a question I would like to inquiry and confirm.
Given a primal dynamic optimization problem (P) in the infinite ...
0
votes
0answers
6 views
What are the conditions for a coordinatewise optimum to be a local optimum
if $x*$ is a coordinatewise optimum of a function $f$, (i.e $x*$ is an optimum of $f$ in all directions), what are the conditions (on the function $f$ or $x*$) for $x*$ to be a local optimum of $f$.
1
vote
0answers
25 views
Using the Newton's method to find the global minimum of a 2D problem with a constraint
I am solving an optimization problem
$$\min f(x_1,x_2)\\
\text{s.t.}~~~~ c(x_1,x_2)\leq 0$$
In my problem, there are two min and two max and I am looking for the global min.
I know that with the ...
0
votes
0answers
32 views
Defining a Jacobian Matrix
reaching out my post. I have a trouble of defining a Jacobian matrix for my problem. Basically, I have 4 differential equations to be solved.
$\dot x_1(t)=x_2(t),$
$\dot x_2(t)=p_2(t)−\sqrt 2 x_1(t)...
0
votes
0answers
18 views
Standard name for the computation of Lagrange multipliers iteratively by fixing other multipliers?
Dear Optimization Experts,
Background:
I have a convex optimization problem on hand that can be shown in general form as given below
\begin{equation}
\begin{aligned}
& \underset{x \in \mathbb{R}...
0
votes
0answers
30 views
Production time optimization model
I have a problem where I have 2 different types of raw-material. These materials can be processed in two different production-lines which take different amount of time, depending on raw material. ...
0
votes
0answers
15 views
Hyperparameter optimization
How to choose hyper-parameters for optimisation methods in practice? I know that there are hyper-parameter optimisation techniques such as gradient-based or bayesian methods, but for instance it is ...
1
vote
0answers
20 views
Numerical Integration Schemes for a Hemispherical Region
I am struggling with a problem that requires me to numerical integrate a function within a hemisphere. While the function is smooth, it is pretty nasty, extremely oscillatory and I have to calculate ...
0
votes
3answers
21 views
Determining coefficients of a parametrization of an epicycloid given a predefined arc length.
I am trying to determine the coefficient q in the parametrization of a epicycloid which gives me the arc length of 4.25. The parametrization can be glimpsed in my attempt of a solution in the ...
0
votes
0answers
19 views
What are the directions of research in Numerical Optimization?
I have just begun reading in the field of Numerical Optimization.
Are people trying to invent new Algorithms? or proving the convergence of Heuristic Algorithms? and what else?
What are the tools a ...
0
votes
0answers
11 views
Solving multiobjective problem with matlb
I would like to solve a multiobjective problem with matlab with NSGA II procedure.
The problem is a maximization/minimizationf objective functions.
Can someone porvide me this code, and explain how to ...
0
votes
1answer
36 views
Maximise $z = \frac{y}{2x+2y}+\frac{50-y}{200-2x-2y}$ given that $x+y$ is non zero and $x+y<100$. Also, $x\leq50$ and $y\leq50$ and non-negative.
Z is actually a probability function. I am finding where the probability is maximized. But I could find no way how to maximize this function.
Original question is as follows:
Mr A wants to join a ...
0
votes
1answer
47 views
Numerical integration in Finite Element Method (and implementation in Matlab)?
i'm trying to solve the p-Laplace Equation:
\begin{align}
\begin{cases}
\text{div} (\sigma |\nabla u|^{p-2} \nabla u) = f &\quad \text{in } \Omega\\
u = g &\quad \text{in } \partial\Omega
\...
1
vote
0answers
15 views
Encouraging sparse output in optimization problem
I am trying to define an optimization problem:
$$ \min_\theta \sum_{x\in X} L(x, \theta, f_\theta(x),\delta_\theta(x)) + \lambda_s S(\delta_\theta(x)) $$
where $X$ is the dataset, $\theta$ are the ...
0
votes
0answers
7 views
Numerically determining if a critical point is a saddle point in the presence of inequality constraints
I have a constrained optimization problem
$$\min_{\mathbf{x}} f(\mathbf{x}) \quad \mathrm{s.t.}\quad g(\mathbf{x}) = \mathbf{0}, h(\mathbf{x}) \geq \mathbf{0}$$
and need to probe if a critical point $\...
0
votes
0answers
56 views
Minimizing a Gaussian Mixture
I'm trying to find the local minima, if they exist, of,
$$G(r) = \sum_{n=1}^N \beta_n e^{{-(r-r_n)}^2}$$
Such that $r$, $r_n$, and $\beta_n \in \Bbb R^+$ are positive scalars.
Edit:
The $r_n$ are ...
0
votes
0answers
23 views
Bad convex functions
Why are functions of the form $ \ f_{m} (x) = \eta_{p+1}(A_m x) - x_1$ bad, where $\eta_{p+1} = \frac{1}{p+1}\sum_{i=1}^{n}(x_i)^{p+1}$, $A_m = \begin{pmatrix}U_m & 0 \\ 0 &I_{n-m} \end{...
0
votes
0answers
14 views
Schatten-1 norm as matrix constraint
suppose I have a tensor $x \in \mathbb{R}^{n \times 2 \times 3}$. I take the seminorm of $x$ given by taking the Schatten-1 Norm in every $2 \times 3$ slice and then the $\ell_1$-Norm of the resulting ...
2
votes
0answers
31 views
Self-study - Numerical Optimization
Is there any available online course on Numerical Optimization other than NPTEL courses?
I'm also looking for lecture notes or textbooks that are suitable for self-study with lots of examples and ...
0
votes
0answers
21 views
Galerkin Approximation
Let $u_N$ be the Galerkin-Approximation of u and $\dim(V_N)=N$. Suppose that the error $\Vert u_N - u \Vert$ can be developed in a power series $\sum_{i=0}^{\infty}C_i(N^{\alpha})^i$ with $C_i,\alpha\...
0
votes
0answers
17 views
Global convergence of the BFGS method
Please help me prove that
$$\det(B_{k+1})=\det(B_k)\frac{y_k^Ts_k}{s_k^TB_ks_k}$$
This is from the global convergence of the BFGS method (Quasi-Newton methods), where $B_k$ is the approximate ...
0
votes
0answers
26 views
Rate of convergence of the steepest descent method on a general linear function
Can someone help me with the proof of this theorem (it's theorem 3.4 from the Nocedal and Wright book - Chapter 3 - Line search methods) regarding the convergence rate of the steepest descent method ...
0
votes
0answers
17 views
Is a hypergeometric sum the minimum of a “potential” function?
I'm wondering if values of a generalized hypergeometric function can be written as solutions to an optimization problem, like this:
$$_q F_p (a_1, \dots, a_p;b_1,\dots,b_q;x)=\min_{t}\psi(a_1, \dots, ...
0
votes
0answers
24 views
Nonlinear optimization with complex residual and jacobian
I am trying to minimize the following function
$\chi^2=(f(t_i;\vec{p})-y_i)^{H}\text{Cov}^{-1}_{ij}(f(t_j;\vec{p})-y_j)$
where $A^H$ is the hermitian conjugate of A, $f(t;\vec{p})-y$ is a complex ...
1
vote
1answer
35 views
Continuity of supremum of polynomial
Prove: Let $A \subseteq \mathbb{R}$ be a compact set. Prove that the function $f \colon\mathbb{R^{n+1}} \to \mathbb{R}$
$\qquad f(x_0,..., x_n) = \sup_{x\in A} \prod_{j=0}^{n} (x-x_j)$
is continuous....
2
votes
1answer
108 views
Second-Order Taylor Series Terms In Gradient Descent
My machine learning textbook states the following when discussing second-order Taylor series approximations in the context of Gradient descent:
The (directional) second derivative tells us how well ...
0
votes
1answer
26 views
determine matrix and vector to fit regularized normal equation
I hope the title is not too unclear.
I am given a Matrix
$$A\in~\mathbb{R}^{K\times~N},~b\in~\mathbb{R}^{K}$$
and instead of solving the normal equation $min_{x\in~\mathbb{R}^N}|Ax-b|^2_2,~$ an $\...
0
votes
0answers
53 views
Is it impossible that gradient descent converges to strict saddle point?
Suppose $f(x), x \in X \subset R^n$, is a smooth function (or real analytic function) which has only one stationary point $x^* \in X$, ($\nabla f(x^*)=0$), and $x^*$ is a strict saddle point which ...
0
votes
0answers
108 views
Proof of sub-multiplicative property in Frobenius norm for $n\times n$ matrices $||AB||_{F}\leq||A||_{F}||B||_{F}$
I wanted to share with you this little demonstration I got making some homework, it seems to be beautiful for me, also it would be nice to recibe some feedback. I would also like to clear out that it ...
0
votes
0answers
7 views
Branch of optimization that combines stochastic search to numerical optimization
Is there any branch of optimization that combines techniques of numerical optimization with stochastic search?
Like we have gradient descent, but I saw in some classes (like machine learning) the ...
0
votes
2answers
30 views
stopping criteria for mathematical optimisation: objective function target, rather than convergence
Researching stopping criteria for mathematical-optimisation algorithms, any libraries I look at (e.g. matlab, apache commons math) only have iteration limits and convergence criteria (e.g. convergence ...
0
votes
0answers
17 views
trust region optimization conjugate gradient steihaug subproblem
I'm trying to solve by hand the trust region optimization conjugate gradient - Steihaug method from the book Numerical Optimization by Nocedal and Wright - Algorithm 7.2 as shown below. I'm struggling ...
0
votes
0answers
47 views
Finding the Geometric Median of N points in 2D Euclidean space when its X co-ordinate is given.
I have a given cluster of N points in 2-space. The goal is to find the geometric median of this cluster, and the x co-ordinate of the geometric median is known. What is the most computationally ...
1
vote
0answers
39 views
Data sets for linear inequality and equality constrained quadratic optimization.
I'm trying to find some test problems for an algorithm that is solving the problem below:
$$ \begin{array}{ll} \text{minimize} & x^T Q x + px\\ \text{subject to} & Ax + b = 0\\ \text{ } &...
0
votes
1answer
24 views
- Optimization - Standard Grid Search
I'm struck into an portfolio opt. problem and the paper I'm replicating (or, better, trying to) is using a "Standard Grid Search".
Since I never encountered it before, I would like to ask you about: ...
0
votes
0answers
55 views
nocedal and wright singular values bounded away from zero
In the Nocedal and Wright Numerical optimization second edition book, pages 255-256, they state that the Jacobians "$J(x)$ have their singular values uniformly bounded away from zero in the region of ...
2
votes
0answers
23 views
Methods for numerically solving field shapes for multiple permanent magnets.
I am trying to optimize an engineering setup which involves an arrangement of permanent neodymium magnets in 3d space. I am planning on using Houdini to solve the optimization as a simulation, which ...
