Questions tagged [optimization]
Optimization is the process of choosing the "best" value among possible values. They are often formulated as questions on the minimization/maximization of functions, with or without constraints.
14,183
questions
0
votes
1answer
11 views
Find all extremum of $f = x^2 + y^2$ on the set $g = y^4 - y^6 - 3 (x^2 + x^4) = 0 $
Let $f = x^2 + y^2$ and $C = \{ (x,y) | g(x,y) = y^4 - y^6 - 3 (x^2 + x^4) =0, y> 0\} $.
Find all extremum of $f$ on the set $C$.
As usual I computed when $F_x = F_y = F_\lambda = 0$ for $F(x,y,\...
0
votes
1answer
14 views
BFGS in comparison with Gaussian newton
I'm reading about quasi-newton method, and I get the key idea is to find an approximated Hessian Matrix. The most popular one is for sure L-BFGS. But I've got see Gaussian newton method simply ...
0
votes
1answer
16 views
Help with solving an optimization problem, connecting surface area and volume.
The question
A tent is in the shape of a triangular prism.
The triangular ends are equilateral triangles.
The volume of the prism is 40 $m^3$
.
Find the length and width of the rectangular base
...
1
vote
0answers
17 views
How to find the best combination of parameters from a very large sets?
I have a processing logic which has 11 parameters(let's say from parameter A to parameter K) and different combinations of theses parameters can results in different outcomes.
...
3
votes
2answers
72 views
How to solve this? (Argument of the complex number in complex plane)
Let the $z \in C$ s.t. $|z-10i|= 6 $
$\newcommand{\Arg}{\operatorname{Arg}}$
Say the $\theta = \Arg(z)$
Find the maximum and minimum value of the $8\sin \theta + 6\cos \theta$
My trial)
Trying to ...
0
votes
0answers
38 views
Solving the following maximization problem analytically
Given continuous random variables $x$ and $y$ and a constant $\beta$, define a random variable $z$ by $z:=y+\beta x$. Further, define a random variable $t$ as a function of $z$:
$$
t:=z-\frac{A}{2}(z-\...
5
votes
3answers
91 views
Minimising expression with absolute values
Let $x,y,z$ be distinct reals and consider the expression $$L=\frac{(|x|+|y|+|z|)^3}{|(x-y)(x-z)(y-z)|}$$ Find the minimum possible value of $L$ over all $(x,y,z)$.
My work
Using a calculator, the ...
1
vote
1answer
42 views
Matrix minimization Optimization
When I study the output-based optimal control problem, I meet such a optimization problem as
$\min\limits_{K\in \mathbb{R}^{s\times m}} x^TC^T K^T RKCx+x^TBKCx+x^TC^TK^TB^Tx$
where $x\in \mathbb{R}^...
0
votes
0answers
22 views
Submodularity of a concave problem
If we define a set function $H(X)$ with the ground set $G=\{g_1,g_2,...,g_N\}$ as follows
$$ H(X) = \max\limits_{\mathbf{a}}\ f(\mathbf{a})\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\\
{\rm ...
-4
votes
1answer
49 views
Find the minimum value of P [on hold]
With $2z^2+3x^2+4y^2=48$ and $x$ $\ge$ $y$ $\ge$ $z$ $>$ $0$, $x^2$ $<$ $\frac{32}{3}$, $y^2$ $<$ $4$ .Find minimum value of $P$ $=$ $xy+yz+xz$
0
votes
2answers
26 views
Minimize cost function with constraint
I have an optimization question that I'm not sure if I interpreted correctly.
A hospital wants to determine a drug amount in a patient's urine using an instrument/method. The maximum total inaccuracy ...
1
vote
0answers
41 views
Constrained optimization using function of function
Suppose I have the following constrained optimization problem
$$max \quad f(x) \quad s.t. \quad g(x)=a$$
whose solution is denoted by $x^{*}$. I want to prove that this is the solution to the above ...
0
votes
3answers
65 views
Solving shortest squared distance problem
The question is "The shortest squared distance, $x^2 + y^2 + z^2$, from the origin $(0, 0, 0)$ to a point $(x, y, z)$ on the plane $x + y + z = 1$ is"
I don't seem to understand where to start so an ...
1
vote
1answer
17 views
maximizing income and quadratic function
The manager of a $1000$ seat concert hall knows from experience that all seats will be occupied if the price of the ticket is $50$ dollars. A market survey indicates that $10$ additional seats will ...
1
vote
1answer
26 views
Minimising the result of multiple equations
Say we have the equations
$$2b=a$$
$$3c=a$$
$$4d=a$$
$$a\neq b\neq c\neq d$$
How can these be solved to find the smallest possible integer value of $a$ with all the other unknowns as integers? Is ...
1
vote
0answers
54 views
Writing the dual of a linear minimisation problem
I am trying to write down the dual of a linear minimisation problem and I would like your help to double check whether I'm doing it right. I'm following the instruction here .
The original ...
4
votes
1answer
37 views
Generalizing the conjugate gradient like this works?
Given $A \in \mathbb{R}^{n \times n}$, a SPD matrix, and a vector $b \in \mathbb{R}^n$, it is possible to solve the problem
$$\min_x \| Ax - b\|$$
with the conjugate gradient method. Its algorithm ...
0
votes
0answers
16 views
Iterative algorithm to find the optimal solution of multi-variable optimization problem
I have a optimization problem presented as following:
\begin{equation}
\mathop {\max }\limits_{({x_1} \to {x_5})} f({x_1},{x_2},{x_3},{x_4},{x_5})
\end{equation}
where $f(x_1, x_1, x_3, x_4, x_5)$ is ...
3
votes
2answers
45 views
Predict Winner and Optimal Strategy in Substraction Game of 3 Towers
Given the following game, what is the strategy to win?
There are 3 heaps with m, n, k stones respectively. m, n, k must all be >1 and can be the same.
At each turn, each player can take either 1 from ...
0
votes
1answer
21 views
What is bounded by $O(T)$ mean?
I have come across something like $f(T)$ be bounded by $O(\sqrt{T})$ many times in optimization context.
usually, $f(T)$ don't have an explicit form.
If $f(T)$ do have an explicit form, say $f_1(T)=\...
1
vote
1answer
50 views
How to optimize a function with the following constraints by using gradient descent?
I am not currently unfamiliar with a numerical optimization, so I am studying them. What I am wondering is that I'd like to optimize a certain function with the following constraints by using gradient ...
1
vote
0answers
12 views
The upper bound of the effective stepsize of the Adam optimizer
In the paper of the Adam Optimizer, the author states in the section 2.1 that the effective stepsize has two upper bounds: $\alpha \cdot (1- \beta_1) \ / \sqrt{1 - \beta_2}$ in the case $1 - \beta_1 &...
-2
votes
0answers
20 views
How to show that this statement is true in Convex Optimization book (Boyd, Vandenberghe)
Suppose we have a convex optimization problem with no constraint. Then the set defined in equation 5.36 is given as follows $$\mathbb{G}=\{f_0(x)\}$$ where $x$ is in the domain of $f_0$. Now, how to ...
0
votes
0answers
18 views
Transforming an optimization problem constrained onto a subspace to an unconstrained optimization problem
Let $f:\mathbb{R}^n \rightarrow \mathbb{R}$ be continuously differentiable everywhere. Let $\mathcal{C} \subseteq \mathbb{R}^n$ be a subspace and consider the optimization problem $$\min_{x\in \...
3
votes
1answer
51 views
Minimizing Functional on L2 space
Let $X$ be a non-empty Borel subset of $\mathbb{R}$ and consider the finite-measure space $(X,\mathcal{B}(X),\mu)$. Fix $y,f^1,\dots,f^n \in L^2_{\mu}(X)$ and define the objective function
$$
\begin{...
1
vote
2answers
50 views
why with least squares I get a minimum?
I was reading about least squares method and every book I read just said that we can get the minimum value solving a equations system. For example. If I have
$$
Q=\sum(Y_i-\beta_0-\beta_1X_i)^2
$$
...
0
votes
0answers
23 views
Second Order Correction for SQP - Nocedal & Wright 2006
I am implementing an SQP algorithm (Nocedal & Wright, 2006 algorithm 18.3) with a second order correction within the line search, which is discussed in the preceding page (page 544).
The last ...
1
vote
0answers
55 views
Max-min and min-max equivalence for the optimization problem
I have the following max-min problem:
$\underset{{\bf X}}{\max} \underset{k}{\min} \|{\bf A}_k{\bf x}_k\|^2_2 $
where ${\bf X} = [{\bf x}_1, \dots, {\bf x}_K]\in \mathbb{C}^{N \times P}$ and ${\bf ...
0
votes
3answers
72 views
Least value of $x+y+z$ where $ax=by=cz$
The following question is a generalization of the case $a=3$, $b=4$, $c=5$ from a MindYourDecisions YouTube video (which I am not going to actually link here).
Given positive integers $a$, $b$, and ...
2
votes
1answer
33 views
Maximizing a permutation of a quadratic form
Let
\begin{align*}
f_A(x) = x^\intercal A x
\end{align*}
for some positive semi-definite $A \in \mathbb{R}^{n\times n}$. Unlike the standard problem
\begin{align*}
f^* = \max_{x: \|x\| = 1}f_A(x)
\end{...
0
votes
0answers
27 views
HJB with discontinuity at boundary
I have an optimization problem whose value function I denote by $U(x,t)$ where $x\in [0,1]$ is a state variable and $t\in [0,1]$ is time. The HJB equation for my optimization problem when $x<1$ is ...
2
votes
1answer
68 views
Loss functions for Regression task
I am trying to understand the idea of Loss functions For Regression Task perfectly.
I have read many textbooks and articles, and I came up with questions related to this subject.
Several different ...
1
vote
0answers
38 views
Basic solution and linearly independent columns - exercise 2.3 Bertsimas and Tsitsiklis
I am trying to solve exercise 2.3 of the book "Introduction to linear optimization" by Bertsimas and Tsitsiklis, which states:
$\textbf{
Exercise 2.3 (Basic feasible solutions in standard form ...
0
votes
0answers
24 views
Least Square by Lagragian mutipliers with complex variables
First let me put some context to my question.
It is not hard to find a derivation of optimal solution of
\begin{eqnarray}
\text{Minimize } & \phi = \frac{1}{2}\left\|Ax - y\right\|^{2}_{2} ...
0
votes
1answer
28 views
Finding saddle point of indefinite quadratic function
There are lot of material on internet how to find the minimum or maximum solution for quadratic systems:
\begin{equation}
f(\mathbf{x}) = \mathbf{x}^T Q \mathbf{x} + \mathbf{y}^T(A\mathbf{x}...
1
vote
1answer
22 views
Maximum of multivariate optimization always higher than the maximum of univariate optimization: Is there a theorem?
I was reading on optimization and thought of the following proposition:
The Maximum (Minimum) of a function in a multivariate optimization problem is always higher (lower) than the Maximum (Minimum) ...
0
votes
1answer
44 views
Steps needed to arrive at the Lagrange dual function
This question is related to the text on page 222-223 on the book Convex Optimization (By Boyd and Vandenberghe). The optimization problem is as follows $$\min. \log(\det(X^{-1}))$$ $$s.t. a_i^TXa_i\...
0
votes
2answers
41 views
+50
Maximum Entropy Distribution with Constraint
I want to find the solution for the maximum entropy distribution with a cost constraint. The specific problem setup is as follows:
Let $\bf{x}$ be a probability distribution.
Let $\bf{c}$ be the cost ...
0
votes
1answer
44 views
Inequality-constrained least-squares problem
Given the following dual optimization problem:
$$\min_x \|Ax - y\|_2\quad\text{such that}\quad \|x\|_2 \leq r.$$
What is the minimizer?
Given the Moore-Penrose pseudoinverse $A^+,$ it is evident to ...
0
votes
0answers
26 views
What function $f(x)$ gives the maximal $\frac{\int_0^1 \log f(x)\,dx}{\left(\int_0^1 \int_0^1 \min(x,y) f(x) f(y)\,dx\,dy\right)^{\!1/3}}$
What function $f(x)$ gives the maximal of
$$\frac{\int_0^1 \log f(x)\,dx}{\left(\int_0^1 \int_0^1 \min(x,y) f(x) f(y)\,dx\,dy\right)^{\!1/3}}?$$
I've found $f(x) = \sqrt{e}/x$ gives a high value, ...
0
votes
0answers
43 views
Continuous Armed Bandit Problem
I am working on a continuous armed bandit (CAB) problem. I have come up with an algorithm that I want to test. Specifically, I would like to be able to compare it with other algorithms that solve the ...
1
vote
1answer
65 views
Viewing Deep Learning as an optimization problem, and general theorems on Duality.
In optimization problems of the type LP, we have methods like the simplex algorithm. The integer version of the problem is I believe NP-complete, but we know that a solution exists and we can find it ...
2
votes
0answers
18 views
Is Uniqueness of Lagrange Multipliers required for Projected Hessian?
I am reading through Chapter 12 of "Numerical Optimization" by Nocedal and Wright, in which Sufficient Second-Order conditions for local optima in constrained optimisation are discussed. They describe ...
2
votes
2answers
68 views
Where am I wrong in this reasoning?
We know that $\log \det (X)$ is a concave function if $X$ is a positive definite matrix. Furthermore, we know that $\det (X^{-1})=(\det(X))^{-1}$ which means that $\log \det(X^{-1})$ is a convex ...
1
vote
0answers
26 views
Why are the variables in the dual linear program the shadow price?
Good evening.
In lineary optimization, we have primal and corresponding dual programs. It is often said that the variables in the dual program can be interpreted as the shadow price for the ...
0
votes
0answers
61 views
Lagrange multipliers constraining functions to be anti-symmetric
I have a functional that I would like to extremise with the constraint that the solution has to be antisymmetric. I am unsure if my understanding of Lagrange multipliers is correct. The statement of ...
1
vote
1answer
38 views
Difference between functions and Markov chains for estimation
Given two random variables $X$ and $Y$ on two alphabet $\mathcal X$ and $\mathcal Y$, I'm interested in minimizing the expected distortion $\mathbb E[d(X,\hat X)]$ for $\hat X$ taking values in ...
0
votes
0answers
22 views
how to minimize the barrier function?
suppose we have a barrier function to minimize:
$$ \frac{1}{3}(x+1)^3+y-r(-\frac{1}{x+1}-\frac{1}{y})$$
r will be decrease till to nearly 0 after several iteration we take 0.0001 as initial value and ...
0
votes
2answers
39 views
maximum and minimum value of the function $ f(x,y) = x^{2} + y^{2} + xy + 3(x+y) $ at $ A = \left \{ (x,y): x^{2}+y^{2} \leq 1, x+y\geq 0 \right \} $
Find the maximum and minimum value of the function $ f(x,y) = x^{2} + y^{2} + xy + 3(x+y) $ at $ A = \left \{ (x,y): x^{2}+y^{2} \leq 1, x+y\geq 0 \right \} $ and the points at which it is obtained.
...
0
votes
0answers
21 views
Minimizing a summation given the value of a different summation.
How to solve this?
$min \quad \sum_{i=1}^n{a_ix_i} $
$s.t \quad {\sum_{i=1}^{n}{\frac{1}{2^{x_i}}}} = 1 $
$m \geq x_i \geq 0 $, $a_i > 0 $, $n \leq log(m) $
All $a_i$ are given. $x_i$ are ...
