Research Projects

The Python Experiment Suite

Another by-product of my scientific work is a useful tool called the
Python Experiment Suite.
It's an open source software tool written in
Python, that supports scientists, engineers and others to conduct
automated generic software experiments on a larger scale with numerous
features: parameter ranges and combinations can be evaluated
automatically, where different experiment architectures (e.g. grid
search) are available. The suite also takes care of logging results into
files, can handle experiment interruption and continuation, for instance
after process termination by power failure, supports execution on
multiple cores and contains a convenient Python interface to retrieve
the stored results. Configuration files ease the setup of complex
experiments without modifying code, and various run-time options allow
for a variety of use cases.

There is also a little example available in the
Snippets section,
demonstrating how to implement the main methods for the Experiment
Suite.

PyBrain

PyBrain is a modular Machine Learning Library for Python.
The acronym PyBrain stands for "Python-based Reinforcement
Learning, Artificial Intelligence and Neural Networks". Its goal is to
offer flexible, easy-to-use yet still powerful algorithms for Machine
Learning Tasks and a variety of predefined environments to test and
compare your algorithms. PyBrain is open source and I'm one of the
main developers, mostly responsible for the Reinforcement Learning
part. More information on how to download and use it can be found on
the PyBrain website.

Artificial Curiosity

I am currently working on artificial curiosity for robots. The
goal here is to let the robot explore its environment independently to
improve its model of the world. It receives positive feedback whenever
the model improves, thus leading it to situations where it can still
learn something new about the world. Already familiar places quickly
become boring and will be left for more interesting new teritory. You
can learn more about this on Jürgen Schmidhuber's
website about artificial curiosity.

Policy Gradients for Robots

In order to use reinforcement learning on real robots, a
method for dealing with continuous states and actions is needed.
Policy Gradients is one such method, and it works quite well in
our simulations (check out the videos). However, the current exploration
strategy is somewhat inefficient, as it adds some randomness to the
action in each single time-step. I am currently developing a more global
exploration strategy that uses random strategies rather than
single actions. In the videos below, you can see both methods compared
to each other. Or download the 12 minute uncut learning
video (Quicktime Movie format, 30MB).