0
votes
0answers
23 views

Deriving the optimal value for the intercept term in SVM

I was reading andrew ng's machine learning lecture notes on SVM. I came across the following equation (finding the optimal value for the intercept term $b$ in the SVM problem): However, I have no ...
8
votes
1answer
168 views

Stochastic gradient descent for convex optimization

What happens if a convex objective is optimized by stochastic gradient descent? Is a global solution achieved?
0
votes
2answers
78 views

machine learning optimization

I was studying SVM and I am having problems in the conversion of this optimization problem into another : and gamma_hat is defined by I had to paste the images because I was having troubles with ...
2
votes
1answer
137 views

Smooth Hinge Loss Lipschitz Constant

Given the smooth hinge loss $L_\epsilon$ as follows $L_\epsilon(y_i (w^T x_i + b)) = \begin{cases} 0 & y_i (w^T x_i + b) \\ \frac{(1-y_i (w^T x_i + b))^2}{2 \delta} & 1 - \delta < y_i (w^T ...
3
votes
1answer
70 views

What type of convex constraint is defined by SQRT?

Let $A$ be an $n \times n$ positive semidefinite matrix and $\forall k, x_k \in \mathbb{R}^n$. The distance with respect to this matrix is defined as $ \|x_i -x_j\|_A := ...
0
votes
0answers
62 views

How to solve Hinge-loss based training without regularization term?

I'm trying to optimize the following objective function, $$\min_w \sum_{i=1}^{N}loss(y_i,x_i,w)$$ where $x_i$ is the training instance, $y_i$ is it's corresponding class label, $N$ is the number of ...
0
votes
0answers
16 views

Confusion related to least squares

I was reading this paper where they have modeled the ys given some samples xs,ys as The paper states that the above optimization problem is equivalent to a least squares problem. I didn't get how ...
1
vote
0answers
31 views

Confusion related to convexity and concavity of a problem

I was reading this paper http://www.ist.temple.edu/~vucetic/documents/wang11kdd.pdf related to adaptive multi-hyperplane machine for non linear classification In that paper, they have mentioned about ...
0
votes
1answer
235 views

Gradient of log softmax in matrix form

Suppose $J(\mathbf{A})$ is defined as follows $$J=\text{tr}(\log \mathbf{P})$$ $$\mathbf{P}=\frac{e^\mathbf{A}}{\mathbf{1} \mathbf{1}' e^\mathbf{A}}$$ where division, exp and log are taken pointwise, ...
2
votes
2answers
216 views

Gradient descent vs ternary search

Consider a strictly convex function $f: [0; 1]^n \rightarrow \mathbb{R}$. The question is why people (especially experts in machine learning) use gradient descent in order to find a global minimum of ...
4
votes
3answers
703 views

Batch vs incremental gradient descent

I am studying Machine Learning, but I believe you guys should be able to help me with this! Basically, we have given a set of training data $\{(x_1,y_1), x(x_2,y_2), ..., (x_n, y_n)\}$, and we need ...