I built a Twitter bot using Auth0 Webtasks and Data.gov

I wanted to build something small, fun, and socially-engaging. I have been following Darius Kazemi (@TinySubversions) and the community of bot makers at Botwiki (@Botwikidotorg), and I decided a Twitter bot was the way to go. I have also been following the US Executive Order 13792 pretty closely. The order directs the US Department of Interior to review whether to downsize "certain national monuments" or sell their oil and mineral rights for profit. The Department has an open "Opportunity for Public Comment." I thought this would be a good chance to tinker with lightweight sentiment analysis of the public comments.

I needed a good place to host my small project, and Auth0's free Webtask.io offering turned out to be a great fit! I didn't want to sacrifice my time on infrastructure and hosting when I wanted to focus on this bot idea. I got to attend Glenn Block's Auth0 Webtasks workshop at .NET Fringe this year, and he fired me up to do this.

The Auth0 Webtask.io Editor is nice, but I have a set of development tools I like. I also wanted to import npm modules to handle the sentiment analysis and Twitter posting. The Auth0 Webtasks CLI is perfect for this.

I prefer writing ES6, so I'll be adding 'use latest' to the top of my scripts. I want to split my logic into a couple of different modules, so I'll use the CLI to bundle my scripts (-b). While I'm hacking, I also want to have automatically update my webtask when I save, so I'll have the CLI watch for changes (-w). Finally, I have some API keys that I don't want to hard-code, so I'll create a file called .secrets to hold on to those for me. (And add .secrets to .gitignore so it doesn't accidentally get committed!)

wt create -b -w --secrets-file .secrets .

The first of my secrets to go in the .secrets file is my Data.gov API key. (It's free and easy to sign up for, and provides access to other US government data besides Regulations.gov.) Next, I created a new Twitter account called @EO13792Bot and created a Twitter app for it. I copied the Consumer Key and Secret, and the Access Token and Secret into the .secrets file.

The Regulations.gov API client exports a function to get the latest comments for a given docket. (DOI-2017-0002, in this case.) After testing requests against the API, it was necessary to add a timeout to fetch.

The text analyzer exports a function to analyze a single comment. I found sentences with the most "negative" sentiment analysis score were [subjectively] the most interesting. I decided to use the most "negative" sentence from a comment as its "pull quote" candidate for tweeting. The text analyzer also hashes the comment to only consider identical "form letter" comments once.

The comment selector and formatter exports a function that ultimately returns a single comment to tweet. It gets the comments from the Regulations.gov API client and maps each comment through the text analyzer. It takes in a list of hashes from previous tweets to exclude those comments. It also ensures that the pull quote can fit into a tweet.

The tweeter is the main body of the webtask. It initializes the Twitter client, gets the list of hashes out of the webtask context storage, and calls the comment selector and formatter to pick its tweet. Once it successfully tweets, the hash is added to the list in context storage.

Testing this work is as simple as hitting the webtask's endpoint in a browser. However, I wanted this to run on its own, checking for comments and tweeting every 10 minutes. I set up the webtask on a schedule with a simple change to the CLI command:

wt cron schedule -b --secrets-file .secrets 10m .

In order to make my code a little more usable by other developers and to save myself from having to remember all the switches for the CLI, I added two scripts to my package.json file. Also, I added wt-cli as a dev dependency.

Now @EO13792Bot is tweeting on its own! Following the bot has given me an insight into some of the 300,000 comments that people have made. And I'm ecstatic to spend my time hacking on my bot ideas and trust the infrastructure and hosting to Auth0!