Hello I have a question about reinforcement learning. I was watching RL course by David Silver, and in lecture 6: Value Function Approximation, he says that in reinforcement learning, the data you get can be dependent, like non-iid data. However to my knowledge, we use Markov state to represent the state of agent in reinforcement learning, and a characteristic of Markov state is each state is independent, we don't need to know the state before previous state to make decision, then why is he saying that data can be dependent? Can someone clarify it for me? Thanks


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.