# Stochastic gradient descent for a function of multiple variables?

I realize SGD is used for large data sets where the iterative solution could be approximated by a random sample's gradient instead of the sum over all samples. My question is suppose I have a function of mutiple variables say 'd', i,e just one sample, could one use stochastic gradient descent for just the function? I am asking this because in one of the homework problems in Gilbert Strang's Data science course asks you to compute a single step of gradient descent for a function of two variables, it is explicitly mentioned, full gradient descent not stochastic? I wonder why?

• SGD is originally used as an iterative method for optimizing a function. See Wiki for more information. I don't understand your second question. A practice problem is a practice problem. SGD is a stochastic approximation of GD. Why are you wondering why? – Tab1e Jun 6 at 17:46
• yep the cost function is a sum over all samples, so I get that, I am wondering for a function in closed form such as f(x,y)=x^3(x-y) – Str91 Jun 6 at 17:52
• I think I found it, one could do a co-ordinate descent as well, minimize f(x,y) one variable at a time but that is not SGD by definition. – Str91 Jun 6 at 17:54

To give an example - consider you have a problem with two variables, $$x,y$$ and 10 data points. Then when using SGD you calculate the gradient with respect to $$x,y$$, but at each sub-iteration use only part of the data points to calculate the gradient - let's say 5 data points, then in the next sub-iteration you use the other 5 data points. In a "meta-iteration" you have used all the data. With CD, you will use all 10 data points, but at the first sub-iteration take a step only in the (negative) direction of the gradient with respect to $$x$$, and in the second sub-iteration use the gradient with respect to $$y$$. A "meta-iteration" have used all the variables.