Setting up a custom PyPi server

Introduction

Whilst I’ve got Jenkins to help me build and test and catch common errors like failing unit tests, slipping test coverage and PEP8 violations. I always find that I spot really stupid errors after I’ve released a version into the wild via PyPI. So to cull the comedy of errors that is my release cycle, I’ve decided to setup a private PyPI server using PyPiServer and this post outlines how I set it up.

Overview

I’m going to be setting up the PyPi server in a virtualenv to be controlled by supervisord and set up Nginx SSL virtualhost as a frontend from which I’ll reverse proxy to the PyPi server.

Setting up PyPi

This is nothing to complicated, it’s a simple case of making a directory, creating/activating the virtualenv and installing the package –

# set up the directories
mkdir -p ~/pypi/packages
cd ~/pypi
# set up the virtualenv and activate it
virtualenv venv
. ./venv/bin/activate
# install the payload
pip install pypiserver

Finally, reverse proxy through Nginx

I always like to use Nginx as a reverse proxy rather than exposing services directly to the web. Maybe someone can tell me that I’m wrong, but it does feel like I have a very competant doorman watching over every HTTP request that comes into my server.

Conclusion

I’m now able to deposit “candidate” builds into my index and test them manually for mistakes or bugs that have been missed in the automatic Q&A done by Jenkins. I’m not convinced it’ll make all releases perfect, but I am certain that I’ll not be pushing builds to PyPi that are fundamentally broken.

However it turns out, the more experience I have with taking responsibility for owning and maintaining packages and by continuing to refine Q&A and the release cycle will make me a better developer.