I have a doubt . I read somewhere that gradient descent will diverge if the step size chosen is large. But the gradient descent say using exact line search says chose a step size only if it moves down i.e f[x[k+1]]< f[x[k]]..
what i read which led to this doubt In some slides
Now ideally it choses the direction of negative gradient[this itself says that the direction is towards the inner contours] and the step size is chosen till the point[using exact line search ] where f keeps on decreasing..so at max which will be reached in case of anisotropic or [circular form] it should straight end at the minimum