0
votes
0answers
17 views

Deriving the optimal value for the intercept term in SVM

I was reading andrew ng's machine learning lecture notes on SVM. I came across the following equation (finding the optimal value for the intercept term $b$ in the SVM problem): However, I have no ...
2
votes
0answers
39 views

Simplify huge optimization task (updated)

Shorten and simplified explanation of a problem: We have a sequence of variables from R of length S : $SEQ = [x_1^{(1)} \ x_2^{(2)} \ x_4^{(3)} \ x_2^{(4)} \ x_{12}^{(5)} \ x_6^{(6)} \ x_{15}^{(7)} ...
3
votes
0answers
38 views

Mathematical analysis of e-shop

I'm ukrainian student, studying applied mathematics in Kiev. I have an online store and some statistics data on it's work. Also I've learned a bit about optimization problems and operation reasearch. ...
0
votes
0answers
10 views

Loss specific inference in graphical models

As far as I have seen, in graphical models, the inference (for training or parameter estimation) is done via maximizing likelihood. While in many applications people need loss specific optimization of ...
6
votes
1answer
162 views

Gradient Descent on Non-Convex Function Works But How?

For Netflix Prize competition on recommendations one method used a stochastic gradient descent, popularized by Simon Funk who used it to solve an SVD approximately. The math is better explained here ...
1
vote
1answer
57 views

Analytical Method to Minimise a Cost Function

I've been researching optimisation methods used to minimise the cost function of a neural network, such as: Back propagation, which calculates the error at each node in order to calculate the partial ...
6
votes
0answers
508 views

Optimization / personalization within clusters

I have the following optimization problem: I have a (random and very noisy) objective function f(A, P), where A is a vector of "observable" parameters of the input and P is the parameters that I can ...
0
votes
2answers
76 views

machine learning optimization

I was studying SVM and I am having problems in the conversion of this optimization problem into another : and gamma_hat is defined by I had to paste the images because I was having troubles with ...
0
votes
0answers
31 views

Online machine learning algorithm for dynamical system

I have a complex dynamical system which takes input as x1, x2, x3 and gives output as y1, y2, y3. I don't have any mathematical model of the system. x(k) is the present input to the system and y(k) is ...
1
vote
0answers
25 views

Is the Support Vector Classifier in some sense optimal?

My question is, is the original hard-margin support vector classifier optimal in some sense? If you have an answer that refers to the soft-margin SVC instead, I'd also be interested. I know that the ...
0
votes
0answers
36 views

Finding an optimal set of weights for combining correlated classifiers

In order to combine classifiers that are correlated with one another, I would need to solve the following optimization problem: Find a vector $\mathbf{w}$ that minimizes $\mathbf{w}^T M \mathbf{w}$ ...
0
votes
0answers
16 views

Confusion related to least squares

I was reading this paper where they have modeled the ys given some samples xs,ys as The paper states that the above optimization problem is equivalent to a least squares problem. I didn't get how ...
0
votes
1answer
64 views

online learning to maximize profit

I have a software which takes input as investment and gives the output as return on a particular stock. Now profit metric $x_i$ is defined as the ratio of return $g_i$ to maximum possible return ...
0
votes
1answer
48 views

SVM - Min square norm

All Support Vector Machine litterature mentions that optimal hyperplane is found as: max 1/∥x∥ (st. constraints) which translates directly to: min ∥x∥ or equivalently min $ ∥x∥^2 $. Here ...
0
votes
0answers
23 views

Equivalent form of Dantzig selector / optimization problem

The Dantzig-selector is the solution to $\min_\beta \|\beta\|_1 \;\;\; s.t. \;\;\; \|X^T(y - X\beta)\|_\infty \leq s$ where $X \in \mathbb{R}^{n\times p}$, $y \in \mathbb{R}^n$, and $\beta \in ...
0
votes
0answers
93 views

Gradient descent/ nonlinear optimization intuition needed

all. I'm taking an introductory AI class, and we're using the gradient descent algorithm to find the optimized/ lowest cost of a set of thetas (variable coefficients) to best fit a regression line. In ...
5
votes
3answers
521 views

Lagrange multipliers and KKT conditions - what do we gain?

I'm working through an optimization problem that reformulates the problem in terms of KKT conditions. Can someone please have a go at explaining the following in simple terms? What do we gain by ...
0
votes
0answers
57 views

Lagrangian Duality Complementary Slackness solution

If $$\alpha^*,\mu^* $$ is the solution of optimalization $$\max_{\alpha, \mu}\mathcal{L}(\omega, \xi, b, \alpha, \mu)$$ How I can show that "complementary slackness condition" $$\alpha_i^* = 0$$ ...
1
vote
0answers
178 views

Gradient Descent for Primal Kernel SVM with Soft-Margin(Hinge) Loss

Given the primal objective $$F({\bf a})=L\sum_{i,j}a_{i}a_{j}k(x_i,x_j) + \sum_{i}max(0, 1-y_i \sum_{j}a_jk(x_i,x_j)$$ for the soft margin SVM, where ${\bf a}=(a_1,...,a_N)$, N being the number of ...
0
votes
1answer
168 views

Subtraction of slope in gradient descent

In the gradient descent algorithm say $f(x)$ (quadratic function) is the objective function. SO the algorithm is defined as $$x_i = x_i - a\frac{\partial f(x)}{\partial x_i}$$ I Just dont quite ...
2
votes
2answers
207 views

Gradient descent vs ternary search

Consider a strictly convex function $f: [0; 1]^n \rightarrow \mathbb{R}$. The question is why people (especially experts in machine learning) use gradient descent in order to find a global minimum of ...
4
votes
3answers
653 views

Batch vs incremental gradient descent

I am studying Machine Learning, but I believe you guys should be able to help me with this! Basically, we have given a set of training data $\{(x_1,y_1), x(x_2,y_2), ..., (x_n, y_n)\}$, and we need ...
1
vote
1answer
272 views

how to find the input for this optimization problem?

Suppose I have a neural network, with input variables $a,b,c,d,f,g$ and output variables $m,n,o,p,q$. Given different input values, the neural network will output corresponding $m,n,o,p,q$. Now I ...