Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Does anybody know the notion of "local variance" of Markov decision processes? Any reference would be appreciated.


share|cite|improve this question
I haven't heard about that - but would you link me to a reference where you saw this notion? – Ilya May 30 '12 at 12:51
I saw once, but I simply forgot it; this is why I asked. – user29271 May 30 '12 at 14:16

The beauty of Google is that you can search for "local variance of markov decision processes" and find what you need. Humorously the first hit was this question on StackExchange. But I found this paper and you can see from the abstract that local mean and local covariance are discussed in it.:

Optimal Control of Stochastic Hybrid Systems Based on Locally Consistent Markov Decision Processes Xenofon D. Koutsoukos

Abstract This paper applies a known approach for approximating controlled stochastic diffusion to hybrid systems. Stochastic hybrid systems are approximated by locally consistent Markov decision processes that preserve local mean and covariance. A randomized switching policy is introduced for approximating the dynamics on the switching boundaries. The validity of the approximation is shown by solving the optimal control problem of minimizing a cost until a target set is reached using dynamic programming. It is shown that using the randomized switching policy, the solution obtained based on the discrete approximation converges to the solution of the original problem.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.