I started learning machine learning and got stuck at the following questions:

  1. Why do we need to iterate the gradient descent algorithm?

  2. Why don't we equate the gradient to zero and find all local minima?

Most likely, we can't reach the minimum; we can just come as close as possible and the learning rate controls how close. Am I right? Or do I miss something?

Sorry if this is a duplicate question. Thanks in advance.


migrated from cstheory.stackexchange.com Jul 30 '18 at 12:18

This question came from our site for theoretical computer scientists and researchers in related fields.

  • $\begingroup$ Generally speaking, finding where the gradient equals zero is only easy for quadratic cost functions. Solving systems of polynomial equations is not easy. $\endgroup$ – Rodrigo de Azevedo Jul 26 '18 at 18:08
  • $\begingroup$ @RodrigodeAzevedo, thanks for reply first of all! but why we can't use Laplace transform in that case? I mean it could take much less computing time $\endgroup$ – Anton Jul 30 '18 at 9:14
  • $\begingroup$ Laplace transform? Where are the differential equations? $\endgroup$ – Rodrigo de Azevedo Jul 30 '18 at 12:36
  • $\begingroup$ @RodrigodeAzevedo, sorry, I might misunderstand you. I thought, that when we find a derivatives from MSE function, we are getting system of differential equations and in case it is difficult to solve it, we might use Laplace transform. $\endgroup$ – Anton Jul 30 '18 at 12:46
  • $\begingroup$ Take a look at this. $\endgroup$ – Rodrigo de Azevedo Jul 30 '18 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.