In this chapter we introduce the problem that we try to solve in the rest
of the book. For us, this problem defines the field of reinforcement
learning: any method that is suited to solving this problem we consider
to be a reinforcement learning method.

Our objective in this chapter is to describe the reinforcement learning
problem in a broad sense. We try to convey the wide
range of possible applications that can be framed as reinforcement learning
tasks. We also describe mathematically idealized forms of the
reinforcement learning problem for which precise theoretical statements can be
made. We introduce key elements of the problem's mathematical structure, such
as value functions and Bellman equations. As in all of artificial
intelligence, there is a tension between breadth of applicability and
mathematical tractability. In this chapter we introduce this tension and
discuss some of the trade-offs and challenges that it implies.