Posts tagged: aws lambda

Lambda Tools is a new project that I’ve been working on over the past few weeks or so. It is a build and deployment toolkit, written in Python, to make it easier to work with AWS Lambda functions. It started out as a user story in our backlog at work but since then it’s grown into a full blown open source project with a life of its own.

The “serverless” model offered by AWS Lambda is a useful and potentially cost-effective one, especially for scheduled tasks that only need to run every so often and don’t require a lot of resources. The “free tier” doesn’t expire at the end of your initial twelve month trial, which is an added bonus.

The downside is that it can be tricky to work with. If your function requires additional libraries, it starts to get a bit more complex, and if any of these are written partly in C and require special compilation, things can get pretty messy. On top of that, you will want to write unit tests for your functions and set up some sort of Continuous Delivery pipeline for them.

Lambda Tools includes several features to make these things easier. For example, it gives you an option to build your function in a Docker container to avoid these messy “invalid ELF header” errors. You configure it simply by creating a YAML file called aws-lambda.yml, which might look like this for example:

Integration with triggers such as CloudWatch cron jobs, API Gateway, and so on

A full-blown sample application

The ability to include or exclude specific files when building your package

At the moment it only supports Python 3.6 but nevertheless it is in a usable state. You can install it using pip install lambda_tools and it includes full instructions in the readme file on the GitHub repository. Additionally, if you’re interested in getting involved with its development, feel free to fork the project and send me a pull request or three. If it’s anything more complex, raise a ticket on the GitHub issue tracker and we’ll chat about it there.

This worked fine when I ran it from the command line, but when I ran it from inside the lambda, one particular instance stubbornly refused to start, even though the lambda ran without errors.

It turned out that the problem was a permissions issue. This particular instance had an additional encrypted EBS volume attached. The call to start_instances() was failing silently.

To fix this, make sure that the role under which your code runs is granted the kms:CreateGrant permission.

It took me a bit of trial and error to figure out which permission to add, but I wanted to make sure I got this one right. You should never give your code any more permissions than the bare minimum it needs in order to do what it needs to do. Unfortunately, figuring out exactly which permissions your code needs to run can sometimes be a bit of a challenge…