0
votes
1answer
98 views

Gradient descent (with line search) for convex functions viewed as alternation

I have fundamental confusion about gradient descent (with line search) and the reason it works. I try to explain my view here, and please tell me where it goes wrong. Let $f: \mathbb{R}^n \to ...
0
votes
0answers
30 views

Why for this equation, convergence is not guaranteed with steepest descent methods?

I uses soft svm recently. I found a good slide from http://www.cs.berkeley.edu/~jordan/courses/281B-spring04/lectures/lec6.pdf. For the soft svm, suppose there are $n$ samples. $f(x_i)=w^Tx_i+b$, ...
2
votes
1answer
31 views

Confusion related to explanation of convexity of a function

I was reading this paper where the define an optimization problem as where K and L are kernel matrices and $\pi$ is the permutation matrix. They have explained that the function is convex because ...
1
vote
1answer
82 views

Convergence rate when solving L1 regularized optimization via coordinate descent with tiny step?

Wondering if there is an established result for the convergence rate when solving L1 regularized optimization via coordinate descent with tiny step? By "tiny step" I mean the step is always set to a ...
0
votes
0answers
127 views

Does Frank Wolfe algorithm always converge for concave functions?

I have implemented Frank-Wolfe algorithm in Python on my Ubuntu 11.10 machine. Does the Frank-Wolfe algorithm always converge for a concave function? My independent variable is of 20 ...