Does anybody know the notion of "local variance" of Markov decision processes? Any reference would be appreciated.
The beauty of Google is that you can search for "local variance of markov decision processes" and find what you need. Humorously the first hit was this question on StackExchange. But I found this paper and you can see from the abstract that local mean and local covariance are discussed in it.:
Optimal Control of Stochastic Hybrid Systems Based on Locally Consistent Markov Decision Processes Xenofon D. Koutsoukos
Abstract This paper applies a known approach for approximating controlled stochastic diffusion to hybrid systems. Stochastic hybrid systems are approximated by locally consistent Markov decision processes that preserve local mean and covariance. A randomized switching policy is introduced for approximating the dynamics on the switching boundaries. The validity of the approximation is shown by solving the optimal control problem of minimizing a cost until a target set is reached using dynamic programming. It is shown that using the randomized switching policy, the solution obtained based on the discrete approximation converges to the solution of the original problem.