We are not done with a environments yet. pycolab is a highly-customisable gridworld game engine with some batteries included that allows you to create your own gridworld games to test reinforcement learning agents.