How to create a Python package - Part 1

In the past I always was not really sure how to properly create a python package. I somehow got everything to work but it often was a mess and I wasn't sure if I did something horribly wrong. So this blog post will outline the process of creating a python package from start to finish. It serves as a tutorial for you and a great reminder for me. I always had to browse through lots of blog posts to find everything I need so now I want to collect the basic things in one post. If you follow these instruction your package will

have a GitHub repository

be listed on PyPi

have automated builds and testing with Travis CI

be easily testable locally

properly handle logging

Prerequisites

This is a list of things that you should own or have knowledge of before we begin.

Step 1: Create a new Github repository

The first thing that you should do when starting a new project is creating a new GitHub repository. This will serve as a central point for storing your code. You can easily work with multiple machines and you do not risk losing your progress if your machine fails. You have versioning and issue tracking and so on. So, head over to GitHub and create a new repository.

Create a new GitHub repository

Choose your package name and license (optional)

In the next step you will need the URL to your new repository. It should look like this: http://github.com/yourAccount/yourPackageName

Step 2: Syncing with travis

Travis CI is a tool that will automatically build and test your package. And the best thing is: it is integrated with GitHub and free. You just have to go to travis-ci.org and log in with your GitHub account. Normally a sync with GitHub will start automatically and it often takes some time. If that does not happen you can manually trigger the sync in your account settings. There you can also activate Travis CI for your package. The Travis CI URL for your package is of the following form: http://travis-ci.org/yourAccount/yourPackageName

You have to switch on Travis CI for your package

Step 3: the folder structure

One thing I really struggled with for a long time is a proper folder structure for the python package. And I know that there is not the one perfect solution. Nonetheless I will present the folder structure that I used for my recent project soccer.

The cool thing is, that you do not have to create the folders yourself. We will use the tool cookiecutter to create it. I assume that you have some kind of workspace or git folder on your machine that serves as an entry point for your projects. Now follow these steps:

Clone the following repository: https://github.com/kragniz/cookiecutter-pypackage-minimal

git clone https://github.com/kragniz/cookiecutter-pypackage-minimal

Create a new cookiecutter project using the pypackage-minimal cookiecutter

If you refresh the homepage of your package you should already see a nice new readme.

Step 5: Get Travis CI working

To get Travis CI to work you need to add a .travis.yml file to your repository. It will tell Travis CI what to do. More about the file here. I will now show you my .travis.yml file and tell you what each part is for. Here is the file:

First we specify python as the programming language (Travis CI can be used with a lot of different programming languages) and the version. I personally use Python 3.6 at the moment. The install command tells Travis CI that it has to install some requirements. For example my package footballdataorg uses the requests package. Read more about the requirements.txt here. For the moment you can add an empty file called requirements.txt to your package. The script command tells Travis CI to run pytest after installing the package. More about pytest later. In short: you can write tests that will check if your code base still works. The last part is optional and you do not necessarily need it. Travis CI sends you a lot of mails when it tries to build your package. It changed the settings such that I only get a mail if the build failed.

After adding, commiting and pushing the two files (.travis.yml and requirements.txt) to your repository your first Travis CI build should be successful. You can identify this by looking at the little batch in your readme file.

Summary

This finishes the first part of my tutorial on creating a python package. You learned how to create a the python package and connect it to GitHub and Travis CI. And Travis CI has already successfully installed your package. In the next part we will look at the setup.py file and learn how to upload the package to PyPi. We will talk about testing and logging, too.

Feel free to comment or tweet at me if you have a suggestion on how to improve my process of creating new python package.