Website SMS Alerts with the Plushcap Python Package: Part 1

When your website crashes you can either hear about it from your alerting software or an angry customer. That’s why a variety of hosted services exist to monitor websites.

However, for small websites you often want a simple open source project that can be set up in a couple of minutes to get the monitoring job done. In this post we’ll walk through the creation of a monitoring and alerting Python package.

This project is called Plushcap. What’s a Plushcap? It’s a species of tiny bird found in South American countries such as Argentina and Peru.

We’re going with the name Plushcap for a few reasons. First, the name is available on PyPI so there will not be any naming conflicts. Second, there are no projects named “plushcap” on GitHub in any language so we won’t be stepping on anyone else’s open source project name. Third and most importantly, I’m a huge fan of the amazing Pelican static website generator. Plushcap is a nod to that other P-bird-named open source project.

Many blog posts just walk through coding an app. This post will go a step farther and walk through the entire lifecycle of a Python open source project from initial code structure to making Plushcap available on PyPI.

Setting up our Initial Python Package

There’s some boilerplate code required for a Python project to be installable via PyPI. We can use the wonderful cookiecutter tool to get our initial codebase together. Cookiecutter has some external dependencies such as Jinja2 so I create a separate virtualenv just for these initial bootstrapping steps.

1

$virtualenv--no-site-packages~/Envs/cookie

Next we activate the virtualenv.

1

$source~/Envs/cookie/bin/activate

Install cookiecutter.

1

$pip install cookiecutter

Now let’s create a directory where our project will be stored along with the requisite files for a Python package. Our input is bolded.

What did cookiecutter just do? It cloned the cookiecutter-pypackage.git repository and inserted the input values into files in our Python package directory named plushcap. We now have a directory with the following files.

AUTHORS.rst – reStructuredText file with a list of package author’s names and optionally their email addresses

CONTRIBUTING.rst – guidelines for open source contributions to this project

HISTORY.rst – placeholder file for documenting changes for each version of the library

Makefile – convenient shortcut commands with the make program for packaging, testing and building the package and documentation

LICENSE – boilerplate legalese with the BSD license

MANIFEST.in – enumerates the files to include in the Python package distributable when it is built

README.rst – the first documentation file for others to review when they encounter the project

requirements.txt – application dependencies for developers working on the Plushcap library

.travis.yml – configuration file for Travis CI automated builds that are free for open source projects

.gitignore – Python-specific Git ignore file for files we do not want to include in the package’s Git repository

We also have the following directories.

docs – documentation files for the project that can be uploaded to and hosted on Read the Docs

plushcap – directory for source files

tests – location for files to test the package

Next our new project will need its own virtualenv.

1

2

$virtualenv--no-site-packages~/Envs/plushcap

$source~/Envs/plushcap/bin/activate

Once the virtualenv is activated we’ll have a prompt like the following to let us know the virtualenv that is currently in use.

1

(plushcap)$

Initial Git commit

For the remainder of this tutorial assume that we’ve activated the plushcap virtualenv even if it is not shown at the prompt. Next we change into the Plushcap project directory.

1

$cd~/devel/py/plushcap

Let’s initialize the git repository and prep a push to a remote repository we’ll set up on GitHub.

Note that on Windows Git may add a remote repository after the git init command. Remove the incorrect remote repository with git remote remove origin then execute steps 2-4.

1

2

3

4

$git init

$git remote add origin git@github.com:makaimc/plushcap.git

$git add-A.

$git commit-m"Initial commit with skeleton project structure."

Let’s get our public repository set up on GitHub so we can push it there.

Selecting “New repository” will send us to a form where we fill in the repository details.

We now have an empty repository after filling in the details and pressing the green “Create repository” button.

Let’s push the local repository to our remote repository.

1

$git push-uorigin master

Now we’ll see the following code in the GitHub remote repository when we refresh the browser window.

We now have the skeleton of a Python package we can now fill our code into. You can see the results at this step or start from here with the tutorial-step-1 tag.

Writing the monitoring code

Let’s write some code to monitor if a website at an arbitrary URL is up or down. We’ll use the Requests library to easily retrieve the HTTP status code and content from URLs we want to monitor. We can then wrap conditional logic around responses to complete a simple version of our Plushcap library.

Requests can be installed through pip.

1

$pip install requests

We also need to make sure Requests is in our requirements.txt file. We can freeze our current requirements with the freeze command.

1

$pip freeze>requirements.txt

Let’s take a look at what’s in there so far. This list is from Linux so if you’re running through these commands on Windows or Mac OS X you may have more or less packages listed. As long as Requests is in there you’re in good shape.

1

2

3

4

$more requirements.txt

argparse==1.2.1

requests==2.3.0

wheel==0.23.0

Wheel was installed by the cookiecutter package. Requests relies on argparse so that’s included as a dependency as well.

Let’s add some code to our library so it’s not just a skeleton structure package. Open a file named plushcap.py under the plushcap/ directory and add the following code into it.

Python

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

importsys

importrequests

CONNECTION_ERROR=-100

responses={CONNECTION_ERROR:"is down or did not respond to the request.",

200:"is online and returning 200 OK.",

403:"is online but is denying the request due to lack of "+\

"permission.",

404:"is online but the webpage at that URL was not found.",

500:"is online but returning an internal server error.",

}

defcontact_url(url):

"""

Attempts to access the URL specified as a parameter. Returns the

status code and the content for the request once it is complete.

"""

try:

response=requests.get(url)

returnresponse.status_code,response.content

exceptrequests.exceptions.ConnectionError:

returnCONNECTION_ERROR,""

The contact_url function works but let’s leave it deliberately simple for this first iteration. We’ll add additional handling and functionality as we go along.

Let’s also add a main function so we can execute Plushcap from the command line.

Python

1

2

3

4

5

6

7

8

9

if__name__=='__main__':

iflen(sys.argv)<2:

print("usage: python plushcap.py http://test.url/")

else:

status_code,content=contact_url(url=sys.argv[1])

ifresponses.has_key(status_code):

print(("The server at %s "+responses[status_code])%sys.argv[1])

else:

print("Server response was unknown, status code: "+status_code)

We can now run the code from within the plushcap project subdirectory. The code will reach out to the URL specified as an argument and find out the HTTP status code. Based on the status code a message will be displayed whether the website is up, down or erroring out.

Testing functionality

Next we’ll add a few tests to ensure a basic scenarios work in this package. The tests will be run before every check in to ensure the main URL retrieval didn’t break with our changes. We’ll add more extensive tests and check our code coverage in a later step.

For now, we have two tests. The first test retrieves a known working URL for Full Stack Python. The second test attempts to contact a localhost server at a port which is unavailable. Add these to the tests/test_plushcap.py file.

Adding notifications

At this point we have a concise Python package that reaches out to a URL and prints a message based on the HTTP status code received. However, we’d like to monitor the URL on a regular interval and know if something happens to the website at the monitored URL. This step is where we can flesh out the code and integrate Twilio’s API to send SMS alerts when the website does not respond or returns an HTTP error status code.

Let’s add a new function to plushcap/plushcap.py that checks a URL at a regular frequency then sends an alert if the status code returned is not 200.

Python

1

2

3

4

5

6

7

8

9

10

11

12

13

defcheck_url(url,twilio_client,twilio_from_number,alert_number,

frequency=60):

"""

Loops in a separate thread checking a given URL. Sends a

notification if the website goes down, 500 errors or gives a 404.

"""

whileTrue:

status_code,content=contact_url(url)

ifstatus_code!=200:

# alert time

send_alert(url,status_code,twilio_client,twilio_from_number,

alert_number)

time.sleep(frequency)

Now let’s create another function that sets up the Twilio client and is the entry point when you set the Twilio environment variables for the Account SID, auth token, from number and alert number.

These values that need to be set for these environment variables are found on the dashboard of your account when you log in or in your account settings page. If you already have a Twilio account you can skip the following sign up steps. If not let’s quickly walk through the registration process.

Packaging and Uploading to PyPI

Now that we have the initial functionality for the package along with tests and documentation we can upload the library to PyPI. We need a MANIFEST.in file that tells setup.py what to include when we build a distributable package. Here’s our MANIFEST.in for Plushcap that sits in the base directory of the package:

With this in place we can package up our Plushcap project and upload it to the central PyPI hosted packages repository. These commands should be run in the setup.py directory which is the base directory of our project. Note that your package cannot be named the same as an existing package in PyPI. Use PyPI’s search feature to find out what package names are available for new libraries. Once you’ve found an available name edit the package_name value in your project’s setup.py file. Then upload the new package with the following command.

1

$python setup.py sdist upload

If everything goes well with the upload we’ll see output with a Server response (200): OK like the following screenshot (installation output capped for brevity).

Now Plushcap is on PyPI and can be installed by anyone with an Internet connection and pip installed by running the following command…

1

$pip install plushcap

…and it’s installed!

Next up: Building a web application with Plushcap

The initial Plushcap library version is ready to go. But what if you want to use the code through a web application instead of directly through the Python interpreter? In the next post of this series we’ll take a look at using the a Python web application framework to build an application around the Plushcap library.