Lambda

Have you ever tried to do something with CloudFormation but just needed a little something more to get the job done? In those cases I’ve started using CloudFormation custom resources. These resources essentially allow you to extend the use of CloudFormation templates whenever you create, update or delete a stack.

In this tutorial, I’ll show you how you could use these custom resources to provision your DynamoDB tables with some data. This can be useful if you’d like to run additional testing on your AWS environment that requires actual data inside of the environment.

This post was originally appeared on the Pluralsight Blog on February 28, 2017. Be sure to check out my Pluralsight course that can introduce you to AWS Lambda!

Maintaining data warehouses can be a difficult undertaking for any organization. Not only do you have to establish processes and procedures for regularly loading flowing data, you also have to ensure you’re doing it in a way that’s resistant to failure and future errors. In this post, we’ll take a quick look at some of the biggest challenges of maintaining large scale data warehouses, and how AWS Lambda can help.

Recently I was trying to use the psycopg2 libraries for Python in combination with AWS Lambda. My first hint that this was probably overkill was that the function package, when zipped, started exceeding 50MB. AWS console errors quickly reminded me that 50MB is the size limit for Lambda packages.
After correcting a few mistakes (my virtualenv bundled in with my dependencies - Whoops!) and trimming some dependencies I started testing again. This time with a function package just a few bytes under the limit.