transmute 3.1

Overview

The idea behind this module is to support self-updating Python
applications, namely command-line tools. Transmute probes remote
repositories for updated components, fetches updates and adds them to
sys.path, making them available for use in the application.

Components are assumed to be standard Python Eggs. Repositories are then
simple containers for these eggs. Currently PyPI and S3 “folders” are
supported as repositories. (Mostly for testing purposes, local
directories are also supported as repositories).

Under the hood, pkg_resources (from setuptools) is used to parse
and fulfill requirements, based on listings of available eggs obtained
from each repository. Once updated packages are made available, their
modules can be imported or the application can be re-launched with an
adjusted environment to pick up updated modules.

The application writer controls the packages to update, repositories
each package will be grabbed from, and when an update is actioned.

The philosophy has been that an absent or flaky network should not
prevent (or significantly delay) an application from running on top of
outdated packages, if they’ve been cached locally. That said, there are
currently no provisions for testing and verifying a successful update or
rolling back a failed update.

Example

This is a script that requests the ‘hello’ package to be updated from
the dist folder in the current working directory:

The script provides a bunch of hooks where users can place their code.
In particular, main() can be filled in to fetch application specific
packages and actually launch the application. At the point it is called
transmute has been added to sys.path (after downloading from
PyPI, if needed).

It can also be used as a placeholder for a Python module. If the module
itself is available from PyPI, the corresponding package name would be
added to the requirements variable. For other use cases, I sense a
pull request coming :-)

At the time of this writing, the only non-standard dependency of the
script is the pkg_resources module from the setuptools package.
The assumption here is that the module is more or less available
everywhere. If this turns out to be a problem in practice, I suppose the
script could be simplified to not require it.

Supported package formats

Currently, only standard Python eggs are supported. I don’t mind adding
support for other formats, formats supported natively by Python are
preferred.

In that regard, source tarballs look particularly interesting for pure
Python packages, and seem to be more generally available from PyPI.
Unpacking and importing the packages locally could be a way forward.