John Langford, Microsoft Research

Abstract

The story of tabular reinforcement learning is nearly solved at a theoretical level, yet the algorithms coming from this process are typically useless in real-world settings. Real world settings often have a rich observation space, for example with an audio or video sensor. Treating these sensors as observations from a Markov Decision Process rapidly leads to statistical intractability. What's needed is a new model of the world. We've found a new model in a Contextual Decision Process which allows for a rich sensor space and yet still implies statistically tractable learning algorithms. This is the only model of reinforcement learning which allows generalization across any class of functions, exploration, and credit assignment.