Optimization is the process of choosing the "best" value among possible values. They are often formulated as questions on the minimization/maximization of functions, with or without constraints.

learn more… | top users | synonyms (5)

0
votes
2answers
31 views

Why this two problems are equivalent?

I was reading about Support Vector Machines and I found that it's equivalent to solve the problem of maximize this number: $\frac{1}{\left \| w \right \|}$ with to minimize this number: ...
6
votes
0answers
180 views

Construct a matrix of polynomials to optimize condition-like score

I'm a physicist currently working on my PhD. Within my studies, my colleagues & I encountered a (strictly mathematical) problem that baffles us (and anyone else we've talked to so far) and is also ...
0
votes
1answer
17 views

Why the dual and the canonical dual may have the same optimal solution?

For instance let be \begin{cases} \max & 3x_1 &+2x_2 &+4x_3 & -x_4\\ &x_1 &+x_2 &-2x_3&&\le4\\ &2x_1&+3x_3&&-4x_4&\ge 5\\ ...
1
vote
1answer
35 views

Converting generic linear problems into their dual

I'm revising how to do dual problems in linear algebra. I'm very weak in Linear programing but I struggle to cope with the topic during lectures and assignements. I have to convert the following ...
0
votes
0answers
38 views

Maximise the function with constraints

Is it possible to maximise this function algebraically $$f(x_{1},x_{2})=5\cdot \min\left(\frac{x_{1}}{6},\frac{x_{2}}{2}\right) + 2\cdot\min\left(\frac{1200-x_{1}}{3},\frac{300-x_{2}}{2}\right)$$ ...
4
votes
4answers
51 views

If $x^2+ax-3x-(a+2)=0\;,$ Then $ \min\left(\frac{a^2+1}{a^2+2}\right)$

If $x^2+ax-3x-(a+2)=0\;,$ Then $\displaystyle \min\left(\frac{a^2+1}{a^2+2}\right)$ $\bf{My\; Try::}$ Given $x^2+ax-3x-(a+2)=0\Leftrightarrow ax-a = -(x^2-3x-2)$ So we get ...
0
votes
1answer
36 views

A Question On Triple Integration

Can anyone construct a nonzero continuous function $f:[0, 1]\times[0, 1]\times [0, 1]\rightarrow [0, \infty)$ such that \begin{equation*} \int_{t_1=0}^1 \int_{t_2=0}^1 \int_{t_3=0}^1 f(t_1, t_2, ...
-1
votes
0answers
16 views

Minimal perimeter of triangle [duplicate]

Given a triangle $ABC$. How one can construct a triangle $DEF$ as $D\in AB$, $E\in BC$, $F\in CA$ and the perimeter of $DEF$ is as short as possible. I found on the net that in acute case the answer ...
1
vote
1answer
45 views

trust region - choice of scaling matrix

According to many resources, TR algorithms often suffer from bad scaling. The simplest remedy is to use scaling matrix D in following way \begin{align} \min_d \ f + g'd + \frac{1}{2}*d'Bd \\ ...
1
vote
0answers
25 views

Find the volume of the largest parallelpiped with faces parallel to coordinate planes $x= 0$,$y = 0$,$z=0$

Question : Find the volume of the largest parallelpiped with faces parallel to coordinate planes $x= 0$,$y = 0$,$z=0$ that can be inscribed in one octant of ellipsoid. I tried making some initial ...
0
votes
0answers
51 views

What values make the solutions in the optimal? infeasible? degenerate? etc

Note that $c_i$'s in the $z_j-c_j$ row are not coefficients of the $x_i$'s. I use instead: $r_1, r_2, r_3$. I'm assuming there's a non-negativity constraint. we need to state necessary ...
0
votes
1answer
51 views

Choose vectors that make this problem infeasible.

Note: $c$ is a row vector. I think the rest are column vectors. $x \ge 0$ means $x_i \ge 0$ What I tried: $$b = [-1 -1], c = -b$$ It seems that both the primal and dual are infeasible. ...
0
votes
1answer
21 views

deriving Newton's method for optimization

I thought I understood the derivation of Newton's method for finding a minimum, but just realized I was not being at all careful! Here are three alternate "derivations". I think the first two are ...
2
votes
2answers
61 views

Maximum value of the sum of absolute values of cubic polynomial coefficients $a,b,c,d$

If $p(x) = ax^3+bx^2+cx+d$ and $|p(x)|\leq 1\forall |x|\leq 1$, what is the $\max$ value of $|a|+|b|+|c|+|d|$? My try: Put $x=0$, we get $p(0)=d$, Similarly put $x=1$, we get $p(1)=a+b+c+d$, ...
1
vote
1answer
37 views

piecewise linear minimization equivalent to linear programming

Why is \begin{equation} \begin{aligned} & \min\max_{i=1,\ldots,n} & &a_i^Tx+b_i\\ \end{aligned} \end{equation} equivalent to an LP \begin{equation} \begin{aligned} & \min & ...
-1
votes
1answer
23 views

Where does the duality comes from in linear programing and can we get the optimal basis from it?

$$\begin{cases} \max & c^Tx\\ & Ax\le b\\ & x\ge 0 \end{cases}\Leftrightarrow \begin{cases} \min & y^Tb\\ & y^TA\ge c^T\\ & y^T\ge 0 \end{cases}$$ Then we come to the ...
0
votes
0answers
8 views

Why $x_B=\tilde b +\tilde A x_{\bar B};c^Tx=\psi+\tilde c^Tx_{\bar B}$ doesn't describe an optimal solution iif $\tilde c_i\le 0,\forall i$

How to counterprove the assertion that if a feasible dictionnary in the type \begin{cases} x_B=\tilde b +\tilde A x_{\bar B}\\c^Tx=\psi+\tilde c^Tx_{\bar B} \end{cases} describe an ...
6
votes
1answer
3k views

Linear Programming: Three variable graphical solution

A small bank offers three type of loans: housing loans at $8.50$% interest, education loans at $13.75$% interest rates, and loans to senior citizens at $12.25$% interest. Further, it needs to ...
0
votes
1answer
13 views

How to shift points optimally for best rounding

I have sets of points. E.g.: 5.664, 2.292, 1.368, 0.18, 3.3, 4.74, 7.812, 6.564, 5.352, 4.008, 2.568, 5.352 I'd like to shift them a bit (add some uniform dx to all of them) to make them closer to ...
0
votes
1answer
1k views

How do I construct the Jacobian for use in a Levenberg-Marquardt algorithm.

I am working on a 3D reconstruction system and I am looking to use a Levenberg-marquardt algorithm to do bundle adjustment. I am not too sure about how LM works and what it requires. The model I am ...
1
vote
0answers
12 views

Infinite Horizon Transversality Condition

I am an economics student, and I have run into a question where I must apply a transversality condition in order to prove that we have a balanced growth path (all variables grow at the same constant ...
2
votes
2answers
33 views

Question about the constraint in Laplacian eigenmaps

When calculating Laplacian Eigenmaps, the original paper mentions about the constraint $$y^TDy=1$$ as "removes an arbitrary scaling factor in the embedding". My understanding is that it prevents $y$ ...
0
votes
2answers
427 views

Minimize Frobenius norm with constraints

As a follow-up on my previous question, I would like to solve the following optimization problem: $\min \Vert MA-B \Vert_F^2-x^HMy\;\;s.t.\;\;M^HM=I$ where $A$ and $B$ are $N\times L$ complex ...
0
votes
0answers
14 views

Finding a function of a random variable that maximizes some expression

The following problem is part of my studies, so I would prefer hints or suggestions for self-study. Let $v_1$ be a random variable taking values in $[a,b]$ for $a,b\in \mathbb R$ and assume that the ...
0
votes
1answer
26 views

Minimum of a bivariate quadratic function

According to (hope my calculation below is correct) https://en.wikipedia.org/wiki/Quadratic_function a bivariate quadratic function is a second-degree polynomial of the form $$ ...
0
votes
2answers
36 views

Optimization of a function over probability distributions

I'm trying to solve certain optimization problems dealing with probability distributions. Consider the space of probability distributions $\{ 1, ..., N\} \to [0, 1]$ I have a function $f : (\{ 1, ...
0
votes
0answers
13 views

optimization problem involving matrices

This optimization problem is confusing me. Assume you are looking for the best matrix ${\bf X}$ and you have a matrix ${\bf V}$. I have the following two optimization problems $${\bf X}^*= ...
0
votes
1answer
31 views

Show that a differentiable function $f:\mathbb{R} \to \mathbb{R}$ has a global max in $a$ if $a$ is its local max

My task is this: Let $f:\mathbb{R} \to \mathbb{R}$ be a differentiable function and assume that the only stationary point $f$ has is a local max in the point $A = (a,f(a))$. Show that $A$ must be a ...
0
votes
0answers
18 views

Optimization under uncertainty, solving for optimal value of $v_1$

I want to solve the following function by finding the optimal value for $v_1$. $$\max_{\begin{array}{c}v_1,\beta_1 \\ 0<\beta_1<1 \\ v_1>0\end{array}} ...
1
vote
1answer
56 views

Maximum of function containing two variables $x$ and $y$

If $x+y+\sqrt{2x^2+2xy+3y^2} = k(\bf{Const.})\;,$ Then $\max(x^2y)\;,$ Where $x,y\geq 0$ $\bf{My\; Try::}$ Let $x^2y=z\;$ Then we get $$x+\frac{x^2}{z}+\sqrt{2x^2+\frac{2z}{x}+\frac{3z^2}{x^4}} = ...
2
votes
0answers
51 views

How to use Expectation Maximization (EM) in Item Response Theory (IRT)?

Could you give a worked example on the steps of Expectation Maximization in Item Response Theory if we use the Two Parameter Rasch Model. The student abilities are unknown and the question parameters ...
2
votes
1answer
680 views

Matlab: need help with optimization

I am trying to minimize the objective function over [x(1),x(2)]: exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1)+b subject to constraint ...
1
vote
2answers
53 views

Optimization with a Probability

Imagine two points in $ℝ^2$ at $(-1, 0)$ and $(1, 0)$. You would like to walk from one point to the next in the shortest distance possible. However, there is a line segment coming from the origin to a ...
1
vote
0answers
37 views

Maximum flow on a directed, acyclic graph

What would be the best algorithm to use for finding max-flow/min-cut on a directed, acyclic graph with integer flows, capacities, and vertex demands? I've been thinking Dinic's Algorithm would be ...
0
votes
0answers
24 views

How to solve exponential matrix factorization with constrain: $UV^T>0$

recently I would like to optimize the following loss function: $$L=\sum_{ij}W_{ij}(X_{ij}-exp(-\sum_{l} U_{il}V_{jl}))^2$$ $$s.t. \sum_lU_{il}V_{jl} > 0$$ Where $W \in \mathbb{R}^{m \times n}, X ...
0
votes
3answers
95 views

Finding the Shortest Distance from Point to Plane

I am trying to find the shortest distance from the point (3,0,-8) to the plane x+y+z = 8 and I keep getting the same incorrect solution. First, I found the equation fo the distance to be: ...
1
vote
1answer
16 views

Solving a polynomial equation along a set of lines numerically.

Assume that I for some reason want to solve multidimensional polynomial equations $$p(x_1,x_2,\cdots,x_k) = 0$$ or possibly (if there is no solution) $$\min_{\forall x_{.}} \{p(x_1,x_2,\cdots,x_k)\}$$ ...
-1
votes
1answer
41 views

Books on Statistics and Optimization

I'm trying to close gaps in my education especially in Statistics and Optimization theory. I had an awful class on Statistics so I want to learn it by myself. As for Optimization we had a pretty good ...
0
votes
1answer
15 views

Is it correct to write $argmin(x, y) \sum_i^n |p_{x_i} - x| + |p_{y_i} - y| = argmin(x) \sum_i^n |p_{x_i} - x| + argmin(y) \sum_i^n |p_{y_i} - y| $?

$argmin(x, y) \sum_i^n |p_{x_i} - x| + |p_{y_i} - y| = argmin(x) \sum_i^n |p_{x_i} - x| + argmin(y) \sum_i^n |p_{y_i} - y| $ Is it a legit way of separating argmins to show independence of $x$ and ...
0
votes
0answers
23 views

Probability of an event occuring $n$ times, given that it can occur $n$ times or does not occur at all.

Suppose you have an event whose probability is $\rho$. This event either does not occur at all or occurs $n$ times, because when it occurs once, all the others occurrences are linked to the first. ...
0
votes
0answers
34 views

Positivity of the last component of non negative least squares based on active set method

I have followed the instructions given in Lawson and Hanson book for non-negative least squares using active set method. I am having a trouble in justifying one of the statements they have made about ...
1
vote
1answer
18 views

Elementary derivation of max/min of quadratic trig polynomial

Let $\alpha, \beta, \gamma, \delta$ be fixed real numbers, and $x$ a variable in $[0,\pi)$. Consider the expression \begin{equation} (\alpha^2+\beta^2)\cos^2(x) + ...
1
vote
0answers
47 views

Why Was Backprop Invented?

I'm currently researching artificial neural networks and I keep wondering why do we use "backpropagation" to train a neural network. An ANN is basically just a very large and complex function ...
2
votes
0answers
87 views

Tricky proof of a result of Michael Nielsen's book “Neural Networks and Deep Learning”.

In his free online book, "Neural Networks and Deep Learning", Michael Nielsen proposes to prove the next result: If $C$ is a cost function which depends on $v_{1}, v_{2}, ..., v_{n}$, he states that ...
1
vote
1answer
306 views

how to find the input for this optimization problem?

Suppose I have a neural network, with input variables $a,b,c,d,f,g$ and output variables $m,n,o,p,q$. Given different input values, the neural network will output corresponding $m,n,o,p,q$. Now I ...
2
votes
1answer
3k views

Why do we use gradient descent in the backpropagation algorithm?

The common approach for training neural networks, as far as i know, is the backpropagation algortihm, which uses gradient descent to reduce the error. (i) why should one use a fixed learning rate / ...
2
votes
1answer
183 views

Gradient descent with adaptive learning ratio.

I have a neural network, trained with SGD (stochastic gradient descent) with learning ratio $\alpha$. Each iteration I try to recalculate the weights with a rule: $$\Delta \vec{w} = -\alpha ...
0
votes
3answers
27 views

Understanding when to use the chain rule when differentiating trig functions.

I'm trying to solve an optimization problem that involves finding the maximum angle that subtends two points. The two points are $b = (0, 5)$ and $t = (0, 14)$. The third point is the point that is ...
-1
votes
1answer
25 views

Periodic point of dynamical system

Hi please help me someone with the proof: We have a function $f:\mathbb{R}\longrightarrow\mathbb{R}$ continous and invertible, discrete dynamical system is given by $x_{n+1}=f(x_n)$ (a): prove that ...
0
votes
1answer
63 views

Show f takes on maximum boundary for function

Suppose $\Omega$ is a bound set in $\mathbb{R}^2$ and $\bar\Omega$ its closure. Assume $f\in C^2(\Omega)\cap C^0(\bar\Omega)$. Moreover, assume $f$ satisfies the partial differential ...