I needed to create a live map of API requests on a map, and the Images returned for theCatApi.com. Not wanting to bloat the API codebase, and wanting to iterate fast - a serverless app was the perfect choice.

The API pushes data into the serverless app using an SQS queue, this allows easy throttling, fault tolerance and scaling, by decoupling the producer from the consumer.

API Gateway new Websocket feature

AWS’s API Gateway has recently added Websocket support, as has the Serverless.com framework. Together this makes a compelling option to build an almost infinitely scalable system to broadcast out messages, with very little development time.

Using the Serverless.com framework I created this stack by describing it in a serverless.yml file. Then finished it off by assigning a custom domain & SSL cert in the AWS console.

If a client disconnects, $disconnect event triggers a Lambda function which deletes connectionId

A new message (Image & location data) from the SQS Queue triggers a Lambda function….

... which gets all the connectionIds from DynamoDB…

… and sends each the message via ‘AWS.ApiGatewayManagementApi.postToConnection()’ using it’s connectionId

Clients receive message via a Websocket and show Image on map to user.

Key points:

Setup API Gateway as Websocket

Assign a Lambda function for $connect, $disconnect, $default

Unique connectionId assigned for each client by API Gateway, picked up in Lambda function via ‘event.requestContext.connectionId’

Save to DynamoDB so the websocket connectionId is available to any other Lambda function wishing to send them messages.

Use AWS.ApiGatewayManagementApi to make callback to each client. As of Apr 2019 it is not in the AWS SDK automatically available in the Lambda runtime, so you have to include it in your bundle, or add it as a Layer.

If AWS.ApiGatewayManagementApi.postToConnection() returns an error with statusCode 410 it means the client has disconnected, so delete the connectionId from the DB

Limitations & things to be aware of:

Despite my best efforts, i was unable to add a path to the url which could be used to filter by topic. This seems to be a restriction. I got around this by having one subdomain for each topic (less than ideal).