1
$\begingroup$

In gradient descent of a function F($\theta_1$,$\theta_2$), in the update step after doing $\theta_1' \leftarrow \theta_1 - \alpha \frac{\sigma F}{\sigma\theta_1}$ when we update $\theta_2$, do we use the old value of $\theta_1$ or we use the updated one $\theta_1'$?

Edit: Do we use the same $\alpha$ step size for both $\theta_1$ and $\theta_2$?

$\endgroup$
  • $\begingroup$ @Sally thank you. just fixed it. $\endgroup$ – user3639557 Feb 18 '16 at 1:12
1
$\begingroup$

Often you will see the update written will all $\theta$s together as $$ \pmb{\theta}' \gets \pmb{\theta}-\mathbf{\alpha}\frac{\partial \mathbf{F}}{\partial \pmb{\theta}} $$ indicating that both happen simultaneously (use the old value) and use the same value $\alpha$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.