Difference between online optimization and stochastic optimization I have come across above two terms often together. Some authors have distinguished one from the other. Can somebody give me precise differences/similarities between  online optimization and stochastic optimization?
Thanks.
 A: Online optimization usually means that the function evaluations or measurements become available sequentially in time and that the algorithm works with the data as it arrives before all data has been collected. I.e., the data collection and optimization occurs simultaneously. For example, think of optimization occurring in a car engine. The performance is measured and used to optimize various objectives online while you drive. However, there need not be any randomness involved in online optimization, although there often are if it is based of measurements.
Stochastic optimization means that some random element is involved in the optimization. This can either be inserted by the algorithm (as in random search methods where random points are tested and the best is selected), or the randomness can be inherent in the function evaluations, e.g. as measurement noise. However, stochastic optimization can be performed in batch, i.e. all data is collected first and the optimization procedure is done offline (for example when fitting least squares).
Stochastic approximation is a special kind of stochastic optimization which is often used for online optimization. Hence, the two are not complementary but overlaps significantly.
