I am new to neural networks and recently found out about gradient descent.
Something does not sit right with me.
Why does this formula work? Wouldn't it make more sense to have lambda a large value thereby mimizing the cost function?
I am not phrasing my question properly as i am honestly quite confused. How could gradient descent result in a global optimum if it always reduces the value?