The mdp package

A library for formulating and solving Markov decision problems.
We currently only solve infinite horizon problems. We handle
discounted and undiscounted problems, and can solve continuous- and
discrete-time problems.