Introduction to stochastic control I'm looking for an introductory text on stochastic control. Any suggestions?
 A: This is also the order I would recommend them in (you will need to find used copies for the first, but that is an excellent text that is accessible and small in size).


*

*Stochastic Systems for Engineers: Modelling, Estimation and Control, John A. Borrie 

*Introduction to Stochastic Control Theory (Dover Books on Electrical Engineering), 
Karl Åström (can peruse on Amazon and price is great)

*Modeling, Analysis, Design, And Control Of Stochastic Systems: 2nd Ed., V. G. Kulkarni (can peruse on Amazon)

*Stationary Stochastic Processes for Scientists and Engineers, Georg Lindgren, Holger Rootzen, Maria Sandsten - this will help you to get your hands around SPs (can peruse on Amazon)

A: I would also recommend the following two books of Kushner which provide a general and rigorous treatment of various topics in stochastic control theory including Lyapunov stability theory:


*

*H.J. Kushner, Introduction to Stochastic Control, Holt, Rinehart and Winston Editions, 1971.

*H.J. Kushner, Stochastic stability and control, Academic Press, New York, 1967.


From the point of view of stochastic programming, I would recommend the following references:


*

*A. Shapiro, D. Dentcheva and A.Ruszczynski, Lectures on stochastic programming - modeling and theory, MPS-SIAM series on optimization, 2009.

*D.P. Bertsekas and S.E. Shreve, Stochastic optimal control: the
discrete time case, Academic press, 1978.


The lecture notes by Shapiro et al. offers a very well-written and comprehensible on stochastic programming. The book of Bertsekas and Shreve provides a very careful and rigorous discussion of dynamic programming for discrete-time stochastic optimal control problems.
If you are interested in the special case of discrete-time Markovian systems, I would recommend:


*

*O.L.V. Costa, M.D. Fragoso and R.P. Marques, Discrete-time Markov Jump Linear Systems, Springer 2005.


This books focuses on discrete-time Markov jump linear systems and discusses various notions of stochastic stability (e.g., mean square stability and almost sure convergence).
I would also recommend the book of Åström - I think it may be a good starting point.
