Skip to main content
1 vote

Convergence of SGD for least squares

First, you can explicitly write down the updates of SGD in this case to be of the form $\theta_{k+1} = A_k\theta_k + u_k$, with $A_k = I - \eta \,x_{i_k}x_{i_k}^T$ and $u_k = \eta y_{i_k}x_{i_k}$. If ...
Orlando's user avatar
  • 105

Only top scored, non community-wiki answers of a minimum length are eligible