Optimization is the process of choosing the "best" value among possible values. They are often formulated as questions on the minimization/maximization of functions, with or without constraints.

learn more… | top users | synonyms (5)

1
vote
0answers
11 views

How to obtain the minimizer parameter $\lambda$ for this computational complexity?

I'm trying to read a certain text, where they reach a computational complexity depending on scalars $a,b,c$ and a parameter $\lambda >0$ $$ O\left(\left\lceil\sqrt{\lambda a + \lambda^2 b^2} ...
2
votes
1answer
36 views

If $H$ is positive definite and $s^Ty>0$, then $s^THs-\frac{s^Tyy^Ts}{s^Ty+y^TH^{-1}y}\ne -1$

Let $H\in\mathbb{R}^{n\times n}$ be symmetric and positive definite $s,y\in\mathbb{R}^n$ with $s^Ty>0$ How can we show, that $$s^THs-\frac{s^Tyy^Ts}{s^Ty+y^TH^{-1}y}\ne -1\;?\tag{1}$$ ...
1
vote
1answer
30 views

Global Optimization, symmetric solutions

Does anyone have the idea to solve the global multivariate minimization problem as below? $$\text{minimizes}\quad (x_1x_2x_3+x_1x_4x_5+x_1x_6x_7+x_2x_4x_6+x_2x_5x_7+x_3x_4x_7)-(x_1+x_2+x_4+x_7) \\ ...
-1
votes
3answers
47 views

How to find a pareto optimal solution in a smart way (3 variables)

$\max\left( { 3x }_{ 1 }+4{ x }_{ 2 }+2{ x }_{ 3 } \right) $ ${ x }^2_{ 1 }+{ x }^2_{ 2 }+{ x }^2_{ 3 }\le 1 $ ${ x }_{ i }\ge 0 $ I have to find a Pareto Optimal solution, but I can't solve this ...
0
votes
3answers
43 views

Shortest distance from a point

Find the shortest distance from the point $(0,b)$ to the parabola $y=x^2+8$. Express your answer in terms of $b$. (Comment: If $b \le \frac{33}{4}$ then the answer is just $|b|$, so assume that $b ...
1
vote
0answers
12 views

Optimizing a multivariate quadratic fn

Let D$\in \mathbb{R}^{m\times n}$, where m$\geq$n & D is full column rank. I'd like to find $\sup_{x\in \mathbb{R}^m}$f(x), where f(x):=$\frac{-1}{2}x^T DD^T x+c^T x$. I know the answer is: ...
2
votes
2answers
336 views

Derive steepest descent vector subject to norm constraint

I am currently working through an old textbook Practical Optimization by Gill, Murray and Wright (c 1982) who make some derivations which seem correct, but I am unable to duplicate. In the equations ...
0
votes
0answers
43 views

Convergence of sequence as $O(1/n)$ using damped iterations

I am trying to understand the argument for convergence here (page 16) for damped iterations of non-expansive maps. Say we have a sequence $\{x^n\}_{n=0}^{\infty}$ that is generated as $x^n = \theta ...
1
vote
2answers
42 views

Maxima and Minima of Functions of Two Variables $ f(x,y) = e^{x+y^2}\cdot y $ and $ f(x,y) = e^{x^2-y^2}\cdot y $

I'm having trouble finding the local minimum and maximum of the next functions: $$1. f(x,y) = e^{x+y^2} \cdot y $$ $ f_x'= (e^{x+y^2}\cdot y) ; $ $ f_y'= (e^{x+y^2}(1+2y^2)) $ $$ 2. f(x,y) = ...
0
votes
0answers
25 views

Pontryagin's Maximum Principle as a sufficient condition?

It is know that Pontryagin's maximum principle provides in general only a necessary condition in the following sense: The ODE system which is known to be solved by the optimal control may have ...
1
vote
1answer
48 views

How to find the minimum of $f(a_1, b_1, \ldots , a_m, b_m) = \sum_{j=1}^{n} (y_j - \sum_{k=1}^{m} a_kx_j^{b_k})^2$?

$$f(a_1, b_1, \ldots , a_m, b_m) = \sum_{j=1}^{n} (y_j - \sum_{k=1}^m a_kx_j^{b_k})^2$$ $$2m < n$$ $x$ and $y$ are constants, and $a$ and $b$ are variables to find. I took deviation out of it and ...
10
votes
3answers
81 views

Global Minimum of $f(a) = \int _{-\infty}^{\infty} \exp\left(-|x|^a\right)dx, a\in(0,\infty)$

Playing around with the Standard Normal distribution, $\exp\left(-x^2\right)$, I was wondering about generalizing the distribution by parameterizing the $2$ to a variable $a$. After graphing the ...
0
votes
1answer
16 views

Extrema function of two variables problem

A rectangular box with a square base is to be constructed from material that costs 9 dollars per $ft^2$ for the bottom, 7 dollars per $ft^2$ for the top, and 4 dollars per $ft^2$ for the sides. Find ...
2
votes
2answers
70 views

Best approximation and an inequality

Let $H$ be a Hilbert space. Let $E\subset H$ and $x\notin E$. Suppose that there exists $y^*\in E$ such that $$\|x-y^*\|=\min_{y\in E}\|x-y\|$$ (i.e., $y^*$ is the best approximant of $x$). I hope ...
0
votes
1answer
24 views

Gradient of a sum of indicators

Say I have a function $\mathbb R^n \rightarrow \mathbb R$: $$f(w_1,\ldots,w_n) = n^-\sum_{i\in I^-}w_ix_i$$ with fixed $x_i\in\mathbb R$ (data), $I^-$ the set of indexes with negative sum operands ...
3
votes
1answer
35 views

Dynamic programming recursion

In a book by Wayne Winston for operations research I found this question. Here's how I did it: Let $t$ be the no.of subjects to pass and let h be the no.of hours she has in hand for studying. ...
0
votes
1answer
14 views

Can a function be both lower (but not upper) semicontinuous and convex?

Is it possible to construct such an example? For example, can a discontinuous function $f : \mathbf{R} \rightarrow \mathbf{R}$ be also convex?
0
votes
3answers
27 views

Extrema Function of $2$ variables [closed]

I do not know how to set this problem up. Any insight as to how to get the equation would be great. It is John's birthday and his parents want to make him a cake in the shape of a rectangular box. ...
0
votes
1answer
18 views

linear programming : Absolute value in constraint in mathematical model

I have a model have an constraint with evaluation of absolute value , a example can be: function objective : $\max \sum(x_i)$ statement: $x_i\geq |(y_i-t_i)|$ for all $i$ but value absolute ...
1
vote
0answers
9 views

Significance of lower semicontinuity in (non-)convex optimisation

In the context of (non-)convex optimisation, what is the reason behind requiring that the objective function be lower semicontinuous? From what I understand, 1) a function is continuous iff it is both ...
1
vote
0answers
54 views

The maximum of a functional

Is the following statement true or false? $$ \max F \left( \theta \right)= \int_{\rho_{min}}^{\rho_{max}} g \left( \rho \right)\pi\left(\rho,\widehat{\theta \left( \rho \right)} \right)d\rho$$ ...
0
votes
0answers
19 views

How to efficiently select a subset of elements that maximizes a certain property? (entropy)

I need to select $k$ elements from a pool containing a much larger number $N$ of elements. The selection must be done in a way that a function $h(\{z_{i_1},\ldots,z_{i_k}\})$ is maximized or ...
1
vote
1answer
17 views

Double integral of a product in calculus of variations

Let's say I have an integral of the form $$ V(u) = \iint\limits_{[0,T]^2}f(x,y)u(x)u(y)\mathrm dx\mathrm dy $$ which I would like to optimize over smooth functions $u$. For the variation I get $$ ...
0
votes
0answers
21 views

Multivariable gradient descent with approximation of gradinet

This is not a statistics problem I have a vector $$X=[x_1,...,x_{10}]$$ and a cost function $$y=F(X)$$ and my aim in to find the best $X$ to minimize the cost function. It is impossible to ...
0
votes
1answer
35 views

Optimization involving the length of $x$ as the perimeter of two geometric objects

I am having problems understanding how to solve the following optimization problem: A piece of wire 12 m long is cut into two pieces, the length of the first piece being x m. The first piece is bent ...
2
votes
3answers
60 views

How to find the sum of distances so that it is minimal?

Question: $A$ and $B$ are two points on the same side of a line $l$. Denote the orthogonal projections of $A$ and $B$ onto $l$ by $A^\prime$ and $B^\prime$. Suppose that the following distance are ...
1
vote
0answers
10 views

How close is $\operatorname{argmax}_p E[\log(f(p,\alpha)]$ to $\operatorname{argmax}_p \log(E[f(p,\alpha)])$?

Here $\alpha$ is a random variable and the expectation is taken with respect to that variable. I am wondering if it's the same in any case or there's a theorem quantifying how close both things are. ...
0
votes
0answers
32 views

What is the minimizer of the matrix norm and it's significance?

For $M_{n\times n}$ a p.s.d real matrix, if we minimize $||M^{\frac{1}{2}}x||_2$ over $x$ under a linear constraint on $x$ as in $Ax=b$, where $b$ is non-zero. what is the significance of this $x$? ...
2
votes
1answer
63 views

Proof of convergence for the proximal point algorithm

I'm trying to come up with a super simple proof of convergence on the proximal point algorithm, which uses the iteration scheme $x^{i+1} = \mathbf{prox}_{tf}(x^i)$ where $f$ is a closed, convex ...
0
votes
0answers
12 views

Is there a minimizer associated with this function in which Hessian is given?

Given a certain function f(x) which we want to minimize the Hessian is found to be: $ \begin{bmatrix} 8&-4\\-4&8 \end{bmatrix} $ And the the point that satisfies the FONC is found to be $ ...
1
vote
0answers
54 views

Derivation of back-propagation equation $\frac{\partial E(\theta)}{\partial W^k}=x*\delta h^k+\tilde{h}^k*\delta y$ for convolutional autoencoders

I was reading the following paper on convolution stacked auto-encoders and they had the following convolution neural network (for auto-encoders, notice I didn't write the offset term [to avoid ...
0
votes
0answers
18 views

Minimize residual Matrix

I have two Matrices $A$ and $B$ of order $m\times n$ and Matrix $C=A-B$ I want to formulate a optimisation problem such that I get the difference at each element should go be minimum, ie, a null ...
1
vote
2answers
440 views

Convex Optimization of quadratic function with inequality constraints

How would I solve the following problem? $$\min_{x\in\mathbb{R}^n} x^T A x$$ subject to the constraints $$x_i\geq 1,\,i=1,\dots,n,$$ where A is positive semidefinite and symmetric. Is it possible to ...
1
vote
0answers
35 views

What is the solution of this optimization problem?

I am looking for the solution of this optimization problem: $$ \min_{x} \max_{1 \leq r\leq N-1} \left|\frac{\sin\pi r M x}{\sin\pi r x}\right|^2$$ where $M \ll N$, $x \in \mathbb{R}$, $r \in ...
0
votes
0answers
31 views

$\mathbf{A}$ is unimodular $\Rightarrow$ $\mathbf{A}$ has entry in $\{-1, 0, 1\}$?

Is it true that $$\mathbf{A}\;\text{is unimodular}\;\Rightarrow\mathbf{A}\;\text{has entry in}\; \{-1, 0,1\}?$$ Also can an unimodular matrix $\mathbf{A}$ has entry in $\mathbb{R}$?
3
votes
1answer
840 views

Linearization of a product of two decision variables

I am trying to solve a problem that involves constraints in which products of two decision variables appear. So far, I read that such products can be reformulated to a difference of two quadratic ...
0
votes
0answers
16 views

How do you compute the weighted sum of data points for learning the centers of a hyper basis function network (HBF)?

I was reading the following paper on hyper basis function (HBF) (similar to radial basis function RBF network) and was trying to figure out how one learns the movable centers of the hyper basis ...
2
votes
4answers
107 views

Optimization with a few Variables (AMC 12 question)

In the 2013 AMC 12B, question 17 says: Let $a$,$b$, and $c$ be real numbers such that $a+b+c=2$, and $a^2+b^2+c^2=12$ What is the difference between the maximum and minimum possible values of $c$? ...
0
votes
0answers
16 views

Maximisation of the distance of particles in a periodic box

Consider $N$ particles in a box of ratio $R=L_x/L_y$, where $L_x$ and $L_y$ are the two sides of the box. The box has periodic boundary conditions. Consider now a state which maximises the distance ...
0
votes
1answer
27 views

Find the minimum number of tanks to hold the maximum quantity of wines, at each tank maximum possible capacity

My business is in the wine reselling business, and we have this problem I've been trying to solve. We have 50 - 70 types of wine to be stored at any time, and around 500 tanks of various capacity. ...
0
votes
0answers
17 views

Notion of outer normal cone and supporting cone if $x \in$ relint($C$)

In my lecture we defined the outer normal cone $ N_c(x^*)= \{ c\ \in \mathbb{R^n} : \max\limits_{x \in C} \ \ c^Tx = c^Tx^* \}$ and the supporting cone $S_C(x^*)= \bigcap\limits_{c \in N_c(x^*)} ...
1
vote
2answers
46 views

Largest Arithmetic Sum of Relatively Prime Numbers Under 30

Pick however many integers in the range $[1,30]$ (inclusive). The only constraint is that all of these numbers must be relatively prime to each other. What is the largest possible arithmetic sum ...
0
votes
3answers
38 views

A lower bound on sorting algorithms

I think I have a proof that $n\ln n$ is optimal in the sense that is it a lower bound for sorting algorithms. See here for a list. It must be greater than $n$ as this is too linear, and the $\ln$ ...
0
votes
1answer
39 views

How can I use Banach Contraction Principle to solve $Ax = b$?

Can anyone explain to me how Banach Contraction Principle (fixed point theorem) makes it easier to solve $Ax = b$?
0
votes
2answers
146 views

Calculus question with optimization homework

A piece of wire 30 m long is cut into two pieces. One piece is bent into a square and the other is bent into a circle. (a) How much of the wire should go to the square to maximize the total area ...
1
vote
1answer
446 views

Using gradient descent and Newton's method combined

I have this function $f(\mathrm{X})$ where $\mathrm{X=A+B+C}$ where $\mathrm{A}$ is a diagonal element with variable $a$ on its diagonal. $\mathrm{B}$ is another diagonal matrix with variable $b$ on ...
2
votes
0answers
17 views

EM algorithm with constrained equation

I am reading a paper where author uses EM for the following equation to find the parameters $\theta$(and $\beta$) : $$ J=\sum_m \alpha_{m}\sum_i\sum_j w_{mij}\log\sum_k \theta_{ik}\beta_{mjk} $$ ...
1
vote
4answers
51 views

Maximum area of rectangle circumscribed about another rectangle.

While studying, I came upon this problem: "Find the maximum area of a rectangle circumscribed about a fixed rectangle of length 8 and width 4." I looked at the answer key, which showed that the ...
0
votes
0answers
44 views

Computational complexity of the following quadratic program (QP)

Let $A^TA$ be a $n \times n$ matrix. I have the following quadratic program to solve: \begin{array}{rl} \min \limits_{x} & x^T A^T A x \\ \mbox{subject to} & \sum_{i=1}^{r} x_i =1, ...
2
votes
3answers
286 views

How to solve $\min \limits_{\mathbf{x}} \| \mathbf{Ax}-\mathbf{b} \|^2$?

Let $\mathbf{x}=[x_1,\ldots,x_K]$. I have the following optimization problem: \begin{array}{rl} \min \limits_{\mathbf{x}} & \| \mathbf{Ax}-\mathbf{b} \|^2 \\ \mbox{s.t.} & x_k\ge 0, \forall ...