Questions tagged [optimization]

Optimization is the process of choosing the "best" value among possible values. They are often formulated as questions on the minimization/maximization of functions, with or without constraints.

Filter by
Sorted by
Tagged with
0
votes
1answer
11 views

Find all extremum of $f = x^2 + y^2$ on the set $g = y^4 - y^6 - 3 (x^2 + x^4) = 0 $

Let $f = x^2 + y^2$ and $C = \{ (x,y) | g(x,y) = y^4 - y^6 - 3 (x^2 + x^4) =0, y> 0\} $. Find all extremum of $f$ on the set $C$. As usual I computed when $F_x = F_y = F_\lambda = 0$ for $F(x,y,\...
0
votes
1answer
14 views

BFGS in comparison with Gaussian newton

I'm reading about quasi-newton method, and I get the key idea is to find an approximated Hessian Matrix. The most popular one is for sure L-BFGS. But I've got see Gaussian newton method simply ...
0
votes
1answer
16 views

Help with solving an optimization problem, connecting surface area and volume.

The question A tent is in the shape of a triangular prism. The triangular ends are equilateral triangles. The volume of the prism is 40 $m^3$ . Find the length and width of the rectangular base ...
1
vote
0answers
17 views

How to find the best combination of parameters from a very large sets?

I have a processing logic which has 11 parameters(let's say from parameter A to parameter K) and different combinations of theses parameters can results in different outcomes. ...
3
votes
2answers
72 views

How to solve this? (Argument of the complex number in complex plane)

Let the $z \in C$ s.t. $|z-10i|= 6 $ $\newcommand{\Arg}{\operatorname{Arg}}$ Say the $\theta = \Arg(z)$ Find the maximum and minimum value of the $8\sin \theta + 6\cos \theta$ My trial) Trying to ...
0
votes
0answers
38 views

Solving the following maximization problem analytically

Given continuous random variables $x$ and $y$ and a constant $\beta$, define a random variable $z$ by $z:=y+\beta x$. Further, define a random variable $t$ as a function of $z$: $$ t:=z-\frac{A}{2}(z-\...
5
votes
3answers
91 views

Minimising expression with absolute values

Let $x,y,z$ be distinct reals and consider the expression $$L=\frac{(|x|+|y|+|z|)^3}{|(x-y)(x-z)(y-z)|}$$ Find the minimum possible value of $L$ over all $(x,y,z)$. My work Using a calculator, the ...
1
vote
1answer
42 views

Matrix minimization Optimization

When I study the output-based optimal control problem, I meet such a optimization problem as $\min\limits_{K\in \mathbb{R}^{s\times m}} x^TC^T K^T RKCx+x^TBKCx+x^TC^TK^TB^Tx$ where $x\in \mathbb{R}^...
0
votes
0answers
22 views

Submodularity of a concave problem

If we define a set function $H(X)$ with the ground set $G=\{g_1,g_2,...,g_N\}$ as follows $$ H(X) = \max\limits_{\mathbf{a}}\ f(\mathbf{a})\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\\ {\rm ...
-4
votes
1answer
49 views

Find the minimum value of P [on hold]

With $2z^2+3x^2+4y^2=48$ and $x$ $\ge$ $y$ $\ge$ $z$ $>$ $0$, $x^2$ $<$ $\frac{32}{3}$, $y^2$ $<$ $4$ .Find minimum value of $P$ $=$ $xy+yz+xz$
0
votes
2answers
26 views

Minimize cost function with constraint

I have an optimization question that I'm not sure if I interpreted correctly. A hospital wants to determine a drug amount in a patient's urine using an instrument/method. The maximum total inaccuracy ...
1
vote
0answers
41 views

Constrained optimization using function of function

Suppose I have the following constrained optimization problem $$max \quad f(x) \quad s.t. \quad g(x)=a$$ whose solution is denoted by $x^{*}$. I want to prove that this is the solution to the above ...
0
votes
3answers
65 views

Solving shortest squared distance problem

The question is "The shortest squared distance, $x^2 + y^2 + z^2$, from the origin $(0, 0, 0)$ to a point $(x, y, z)$ on the plane $x + y + z = 1$ is" I don't seem to understand where to start so an ...
1
vote
1answer
17 views

maximizing income and quadratic function

The manager of a $1000$ seat concert hall knows from experience that all seats will be occupied if the price of the ticket is $50$ dollars. A market survey indicates that $10$ additional seats will ...
1
vote
1answer
26 views

Minimising the result of multiple equations

Say we have the equations $$2b=a$$ $$3c=a$$ $$4d=a$$ $$a\neq b\neq c\neq d$$ How can these be solved to find the smallest possible integer value of $a$ with all the other unknowns as integers? Is ...
1
vote
0answers
54 views

Writing the dual of a linear minimisation problem

I am trying to write down the dual of a linear minimisation problem and I would like your help to double check whether I'm doing it right. I'm following the instruction here . The original ...
4
votes
1answer
37 views

Generalizing the conjugate gradient like this works?

Given $A \in \mathbb{R}^{n \times n}$, a SPD matrix, and a vector $b \in \mathbb{R}^n$, it is possible to solve the problem $$\min_x \| Ax - b\|$$ with the conjugate gradient method. Its algorithm ...
0
votes
0answers
16 views

Iterative algorithm to find the optimal solution of multi-variable optimization problem

I have a optimization problem presented as following: \begin{equation} \mathop {\max }\limits_{({x_1} \to {x_5})} f({x_1},{x_2},{x_3},{x_4},{x_5}) \end{equation} where $f(x_1, x_1, x_3, x_4, x_5)$ is ...
3
votes
2answers
45 views

Predict Winner and Optimal Strategy in Substraction Game of 3 Towers

Given the following game, what is the strategy to win? There are 3 heaps with m, n, k stones respectively. m, n, k must all be >1 and can be the same. At each turn, each player can take either 1 from ...
0
votes
1answer
21 views

What is bounded by $O(T)$ mean?

I have come across something like $f(T)$ be bounded by $O(\sqrt{T})$ many times in optimization context. usually, $f(T)$ don't have an explicit form. If $f(T)$ do have an explicit form, say $f_1(T)=\...
1
vote
1answer
50 views

How to optimize a function with the following constraints by using gradient descent?

I am not currently unfamiliar with a numerical optimization, so I am studying them. What I am wondering is that I'd like to optimize a certain function with the following constraints by using gradient ...
1
vote
0answers
12 views

The upper bound of the effective stepsize of the Adam optimizer

In the paper of the Adam Optimizer, the author states in the section 2.1 that the effective stepsize has two upper bounds: $\alpha \cdot (1- \beta_1) \ / \sqrt{1 - \beta_2}$ in the case $1 - \beta_1 &...
-2
votes
0answers
20 views

How to show that this statement is true in Convex Optimization book (Boyd, Vandenberghe)

Suppose we have a convex optimization problem with no constraint. Then the set defined in equation 5.36 is given as follows $$\mathbb{G}=\{f_0(x)\}$$ where $x$ is in the domain of $f_0$. Now, how to ...
0
votes
0answers
18 views

Transforming an optimization problem constrained onto a subspace to an unconstrained optimization problem

Let $f:\mathbb{R}^n \rightarrow \mathbb{R}$ be continuously differentiable everywhere. Let $\mathcal{C} \subseteq \mathbb{R}^n$ be a subspace and consider the optimization problem $$\min_{x\in \...
3
votes
1answer
51 views

Minimizing Functional on L2 space

Let $X$ be a non-empty Borel subset of $\mathbb{R}$ and consider the finite-measure space $(X,\mathcal{B}(X),\mu)$. Fix $y,f^1,\dots,f^n \in L^2_{\mu}(X)$ and define the objective function $$ \begin{...
1
vote
2answers
50 views

why with least squares I get a minimum?

I was reading about least squares method and every book I read just said that we can get the minimum value solving a equations system. For example. If I have $$ Q=\sum(Y_i-\beta_0-\beta_1X_i)^2 $$ ...
0
votes
0answers
23 views

Second Order Correction for SQP - Nocedal & Wright 2006

I am implementing an SQP algorithm (Nocedal & Wright, 2006 algorithm 18.3) with a second order correction within the line search, which is discussed in the preceding page (page 544). The last ...
1
vote
0answers
55 views

Max-min and min-max equivalence for the optimization problem

I have the following max-min problem: $\underset{{\bf X}}{\max} \underset{k}{\min} \|{\bf A}_k{\bf x}_k\|^2_2 $ where ${\bf X} = [{\bf x}_1, \dots, {\bf x}_K]\in \mathbb{C}^{N \times P}$ and ${\bf ...
0
votes
3answers
72 views

Least value of $x+y+z$ where $ax=by=cz$

The following question is a generalization of the case $a=3$, $b=4$, $c=5$ from a MindYourDecisions YouTube video (which I am not going to actually link here). Given positive integers $a$, $b$, and ...
2
votes
1answer
33 views

Maximizing a permutation of a quadratic form

Let \begin{align*} f_A(x) = x^\intercal A x \end{align*} for some positive semi-definite $A \in \mathbb{R}^{n\times n}$. Unlike the standard problem \begin{align*} f^* = \max_{x: \|x\| = 1}f_A(x) \end{...
0
votes
0answers
27 views

HJB with discontinuity at boundary

I have an optimization problem whose value function I denote by $U(x,t)$ where $x\in [0,1]$ is a state variable and $t\in [0,1]$ is time. The HJB equation for my optimization problem when $x<1$ is ...
2
votes
1answer
68 views

Loss functions for Regression task

I am trying to understand the idea of Loss functions For Regression Task perfectly. I have read many textbooks and articles, and I came up with questions related to this subject. Several different ...
1
vote
0answers
38 views

Basic solution and linearly independent columns - exercise 2.3 Bertsimas and Tsitsiklis

I am trying to solve exercise 2.3 of the book "Introduction to linear optimization" by Bertsimas and Tsitsiklis, which states: $\textbf{ Exercise 2.3 (Basic feasible solutions in standard form ...
0
votes
0answers
24 views

Least Square by Lagragian mutipliers with complex variables

First let me put some context to my question. It is not hard to find a derivation of optimal solution of \begin{eqnarray} \text{Minimize } & \phi = \frac{1}{2}\left\|Ax - y\right\|^{2}_{2} ...
0
votes
1answer
28 views

Finding saddle point of indefinite quadratic function

There are lot of material on internet how to find the minimum or maximum solution for quadratic systems: \begin{equation} f(\mathbf{x}) = \mathbf{x}^T Q \mathbf{x} + \mathbf{y}^T(A\mathbf{x}...
1
vote
1answer
22 views

Maximum of multivariate optimization always higher than the maximum of univariate optimization: Is there a theorem?

I was reading on optimization and thought of the following proposition: The Maximum (Minimum) of a function in a multivariate optimization problem is always higher (lower) than the Maximum (Minimum) ...
0
votes
1answer
44 views

Steps needed to arrive at the Lagrange dual function

This question is related to the text on page 222-223 on the book Convex Optimization (By Boyd and Vandenberghe). The optimization problem is as follows $$\min. \log(\det(X^{-1}))$$ $$s.t. a_i^TXa_i\...
0
votes
2answers
41 views
+50

Maximum Entropy Distribution with Constraint

I want to find the solution for the maximum entropy distribution with a cost constraint. The specific problem setup is as follows: Let $\bf{x}$ be a probability distribution. Let $\bf{c}$ be the cost ...
0
votes
1answer
44 views

Inequality-constrained least-squares problem

Given the following dual optimization problem: $$\min_x \|Ax - y\|_2\quad\text{such that}\quad \|x\|_2 \leq r.$$ What is the minimizer? Given the Moore-Penrose pseudoinverse $A^+,$ it is evident to ...
0
votes
0answers
26 views

What function $f(x)$ gives the maximal $\frac{\int_0^1 \log f(x)\,dx}{\left(\int_0^1 \int_0^1 \min(x,y) f(x) f(y)\,dx\,dy\right)^{\!1/3}}$

What function $f(x)$ gives the maximal of $$\frac{\int_0^1 \log f(x)\,dx}{\left(\int_0^1 \int_0^1 \min(x,y) f(x) f(y)\,dx\,dy\right)^{\!1/3}}?$$ I've found $f(x) = \sqrt{e}/x$ gives a high value, ...
0
votes
0answers
43 views

Continuous Armed Bandit Problem

I am working on a continuous armed bandit (CAB) problem. I have come up with an algorithm that I want to test. Specifically, I would like to be able to compare it with other algorithms that solve the ...
1
vote
1answer
65 views

Viewing Deep Learning as an optimization problem, and general theorems on Duality.

In optimization problems of the type LP, we have methods like the simplex algorithm. The integer version of the problem is I believe NP-complete, but we know that a solution exists and we can find it ...
2
votes
0answers
18 views

Is Uniqueness of Lagrange Multipliers required for Projected Hessian?

I am reading through Chapter 12 of "Numerical Optimization" by Nocedal and Wright, in which Sufficient Second-Order conditions for local optima in constrained optimisation are discussed. They describe ...
2
votes
2answers
68 views

Where am I wrong in this reasoning?

We know that $\log \det (X)$ is a concave function if $X$ is a positive definite matrix. Furthermore, we know that $\det (X^{-1})=(\det(X))^{-1}$ which means that $\log \det(X^{-1})$ is a convex ...
1
vote
0answers
26 views

Why are the variables in the dual linear program the shadow price?

Good evening. In lineary optimization, we have primal and corresponding dual programs. It is often said that the variables in the dual program can be interpreted as the shadow price for the ...
0
votes
0answers
61 views

Lagrange multipliers constraining functions to be anti-symmetric

I have a functional that I would like to extremise with the constraint that the solution has to be antisymmetric. I am unsure if my understanding of Lagrange multipliers is correct. The statement of ...
1
vote
1answer
38 views

Difference between functions and Markov chains for estimation

Given two random variables $X$ and $Y$ on two alphabet $\mathcal X$ and $\mathcal Y$, I'm interested in minimizing the expected distortion $\mathbb E[d(X,\hat X)]$ for $\hat X$ taking values in ...
0
votes
0answers
22 views

how to minimize the barrier function?

suppose we have a barrier function to minimize: $$ \frac{1}{3}(x+1)^3+y-r(-\frac{1}{x+1}-\frac{1}{y})$$ r will be decrease till to nearly 0 after several iteration we take 0.0001 as initial value and ...
0
votes
2answers
39 views

maximum and minimum value of the function $ f(x,y) = x^{2} + y^{2} + xy + 3(x+y) $ at $ A = \left \{ (x,y): x^{2}+y^{2} \leq 1, x+y\geq 0 \right \} $

Find the maximum and minimum value of the function $ f(x,y) = x^{2} + y^{2} + xy + 3(x+y) $ at $ A = \left \{ (x,y): x^{2}+y^{2} \leq 1, x+y\geq 0 \right \} $ and the points at which it is obtained. ...
0
votes
0answers
21 views

Minimizing a summation given the value of a different summation.

How to solve this? $min \quad \sum_{i=1}^n{a_ix_i} $ $s.t \quad {\sum_{i=1}^{n}{\frac{1}{2^{x_i}}}} = 1 $ $m \geq x_i \geq 0 $, $a_i > 0 $, $n \leq log(m) $ All $a_i$ are given. $x_i$ are ...