Thursday, 22 December 2011

mincemeat.py: MapReduce on Python

mincemeat.py is a Python implementation of the MapReduce distributed computing framework.

mincemeat.py is:

Lightweight - All of the code is contained in a single Python file (currently weighing in at <13kB) that depends only on the Python Standard Library. Any computer with Python and mincemeat.py can be a part of your cluster.

Fault tolerant - Workers (clients) can join and leave the cluster at any time without affecting the entire process. (Master checkpointing coming in future versions)

Secure - mincemeat.py authenticates both ends of every connection, ensuring that only authorized code is executed. (TLS support coming in future versions)

Open source - mincemeat.py is distributed under the MIT License, and consequently is free for all use, including commercial, personal, and academic, and can be modified and redistributed without restriction.