I post this question here because I think this is a calculus problem
I'm a software engineer, and I have just started a Udacity's nanodegree of deep learning.
I have also worked my way through Stanford professor Andrew Ng's online course on machine learning and now I'm comparing.
I have a big doubt about Gradient Descent with sigmoid function because on Andrew Ng's course it is different from the one I see on Udacity's nanodegree.
But, from Udacity's nanodegree is (Second formula):
But in this CS229 course notes from Andrew Ng's, on page 18, I have found the demonstration from Andrew Ng's gradient ascent formula. I only add it here as a demonstration because I haven't found this derivative process for gradient descent and I don't know how to do it:
Note: the formula above is for Gradient Ascent.
I'm not sure if I have understood everything, but in this derivative I see have the derivative from
f function disappears (f function is the sigmoid function).
But in Udacity's nanodegree they continue using the sigmoid's derivative in their gradient descent.
The difference between first formula and second formula is the derivative term.
Are the two formula equivalents?
Where can I find all partial derivative steps for that partial derivative?