Python: Creating Your Project Structure

I love using Python for creating command-line applications which require just a bit more logic than writing a bash-script.

Python has excellent libraries for parsing command-line arguments and running other shell commands, while at the same time you can take advantage of the powerful object-oriented language. Additionally you can verify and document your application with the Python's unit testing framework.

Structuring your Application

In my experience the best folder structure for a Python project is to put your executables in a bin folder and your project into a your project name folder (I will use name project in this text). This way you can keep your core functionalities separated and reusable. It is also a standard for other applications.

In project folder you should have main.py as the main access point to your application. Your functionalities should then go to lib and your unitests to tests. This way the skeleton of your project should look like:

+ bin
- project
+ project
- main.py
+ lib
+ tests

Now your application can be executed by running:

$ bin/project <parameters>

Separating Parameters, Shell Commands and Functionalities

As in all object-oriented programming you should aim to separate your concerns. It's often forgotten in Python applications because reading the command-line parameters, handling options and running other shell commands is very easy.

Parsing the Command-Line Options

Create a class which defines and collects the command-line parameters. Python provides optparse, for which you can very easily define the options and behavior:

Now you have created a parser which reads a target value to the example variable by running bin/project -x <target value> or bin/project --example <target value>

Running other Shell Commands

If you want to create an application which depends on other shell commands you should separate the shell execution in its own class. This way your core functionalities can be easily re-used in other environments or applications, and you can more easily collect logs, errors and exceptions that come from external sources. I recommend you to use the Python subprocess for your external shell commands.

The Core Functionalities

In your project/lib/project.py you can now implement the core functionalities. Since this is an example I've only included receiving options(which have been separated from collecting the command-line arguments) and running the date command through your Process-class. See project.py in the example project.

For the person who asked how to build this, it doesn't need to be built. It is run as a script. There are python modules which are built and installed using setup.py as packages. This example is not a package however.