The Foundry @ Cornell Tech

StoryBot - A Facebook Messenger Bot Experiment

27 June 2016

Hot on the heels of the numerous bot-related announcements made this year by the tech giants, I wanted to get my hands dirty with making a chat-bot of my own. The result is StoryBot, a Facebook messenger bot whose purpose is to mediate a simple writing game in which you are paired up with another anonymous person and take alternating turns writing a story. As as player, you have no direct means of communicating with the other person and are also provided with a writing prompt should you choose to follow it.

The high level goal of the project was to go through a product cycle from ideation and development through iteration and testing and finally launching with a smidget of marketing. The project specific goal was to explore the application of bots for purposes different than ordering stuff, scheduling meetings, or looking up the weather which dominate the current bot scene.

Prologue - Ideation

Taking a quick gander at chatbots.org company listings we see a pretty distinct breakdown in chat bot companies falling into one of the following buckets: AI virtual assistants (scheduling meetings, ordering food, etc), customer support, and platforms to make and manage bots. While good and all, I was a bit disappointed at the lack of diversity in the offerings. If “bots are the new apps” as so many rave (i.e. TechCrunch , Economist , etc) then surely there would be room for more diversity in the product offering. If I open up an app store for a platform and just see Virtual Assistant A through Z, then that is a pretty sick ecosystem (in a bad way).

Given my finite time commitment, about a month, I figured I should go completely rogue and try something new. Perhaps the reason these bot companies, some of which are VC backed, are all in the same space may be because that is the only viable space for a company based around bots to realize itself.

Some of the ideas I considered were: RecipeBot, a bot which would take a person through a cooking recipe, perhaps on Amazon Echo; InstructoBot, a generalization of RecipeBot but for arbitrary instruction sets (i.e. lego building instruction); ChowRoullete, a bot which would randomly order a meal for you (yes it’s ordering food, but in a silly fun way); StoryBot, a bot which is a modification of a story writing app I wanted to make a year ago similiar to the ice breaker where you take turns saying a word to form a sentence.

All of these ideas were chosen in the hopes they would complement the bot platform restrictions. Unlike an app a bot has no UI asides from a simple conversation thread. As such the application had to lend itself to something which can be represented as a state system.

In the end, the StoryBot won as it was truly unlike any of the other existing bots as far as I could tell and that excited me. Furthermore I was curious what type of stories, if any, people would come up with and how people would even interact with the bot.

Chapter 01 - If you build it, they will come

Ok, so an idea existed - try to replicate the round-robin story ice-breaker game but in a chat interface. The next question was what bot platform to target. Twilio, the OG platform was the first thought, however I did not really want to spend money on sms fees nor did I think the short message character limit of SMS would lend itself well to the game. I then looked at both the newly announced Facebook and Microsoft bot platforms. Both looked good and well documented, but what sealed the deal for me was Facebook Messenger’s minimalistic UI support - buttons, pictures, cards. These rendered pretty well on the apps and the website and eliminated the need for me to do any fancy text parsing. In addition, the majority of people I would want to play this game with were on facebook messenger rather than Skype.

So going into the Facebook developer docs it appeared all I would need in order to get up and running is A) A facebook page, B) A facebook app, and C) A SSL equipped webhook endpoint. Not too bad, the first two were pretty easy and for the last I just used ngrok to create a temporary SSL endpoint to hit my localhost django app endpoint during development.

From that point on, Facebook just validates that endpoint by requesting a keyword you provide in the settings page, and once validated every time a message is sent to your Page (yes pages now have a Message Me button) the content will hit your webhook endpoint with a POST.

What followed was me tinkering with understanding the Facebook POST payload - an array of events with a sender and recipient identifier (different from their facebook profile id). In order to send a message to a person you just need to provide that id in your POST request to Facebook and it will do the routing for you. The events ranged from an event for when a user has viewed a message in the thread, to when they sent a text, image, etc message.

This made it pretty simple to parse all inputs, and in order to render a button or any of the fancy UIs, you just need to follow the request structure facebook expects. There were a few annoying limitations though such as a 300 character limit to text messages, and a limit to the number of buttons one could add to a single message. These were dealt with by just chunking the responses into the quotas and sending the messages one after the other. Surprisingly I did not encounter or have to deal with any messages being delivered out of order which was a very pleasant surprise.

The Django backend I put together to handle the bot, which can be found here, can be summed up as a very naive state machine. A contributor is either in a browsing or writing state. If they are in a writing state, they have an associated story id which points to the story they are writing. Stories themselves are broken up into a prompt (scraped from Reddit’s Writing Prompts using scrapy), and a series of fragments. Each fragment is just a snippet assigned to a contributor. Initially these were fixed to three fragments per story and I expected everyone to write at the same time and in the end the story will come together, but later this was changed to the round robin style with 4 - 8 fragments per story as it proved to be more fun with early user testing.

A few days later, I had StoryBot v0 deployed on AWS with a letsencrypt SSL ready for testing.

Chapter 02 - It is alive! (But no one gets it)

So there I was, with my cute StoryBot, excited to start writing stories with friends. I shared this facebook page with a few people at Cornell Tech and with a group of 10 facebook friends asking them to try out the bot.

The very first thing I noticed was people had no idea how to even initiate the story. I had seeded the bot with a few stories with a friend of mine in the hopes of showing some examples, but these were misunderstood for the content. Some people thought that “they missed it” and others simply looked at the facebook page and had no idea what to do. Those who were pointed to the landing page of the website had no idea what to do with the Facebook Call to Action button. People just do not have this set of actions under their list of acceptable behaviors for websites and facebook pages and thus were confused as to what was expected.

Then once people understood they can message a facebook page and initiate a conversation, they were unsure of how to actually interact with the buttons. I talked to a few people who I noticed were ignoring the buttons and instead writing commands, and they said that they did not realize they were meant to click on the buttons. Both were on the desktop client rather than the mobile app though, so perhaps it is more understandable on the app.

Furthermore, people complained that the bot prompts were confusing and overwhelming. Initiating a story would result in about 8 messages being sent taking up most of the screen room in a mobile interface and since they went back to back, some people were just overwhelmed by the rate of the messages.

The main takeaways from this initial testing were:
1) People are not familiar with bots yet and their call to actions
2) A bot should be extremely succinct and to the point
3) The bot should always have the final word, some type of option or exit pathway since some of these conversations may be idle for a while and when a person comes back to them they should know what to do without trying to recall some arbitrary command.

To address the first point, I started including a few words on every consumer facing post to emphasize that this happens over Facebook Messenger and how to initiate the chat.

The second point was an exercise in condensing all the bot responses and status updates to only the bare minimal necessary. In addition I started to include flare - or gifs which act as visual anchors in the thread so that when a user browses the thread they can visually parse it without having to read the content. In fact, visual cues were used very strongly to offload textual information.

So for example, every time a player starts a conversation with StoryBot for the first time they receive the following gif.

This gif is accompanied with a small sentence describing the game, but the visual imagery alone shows how each story is made from a collection of chat messages. A similar approach was taken for when a story is done.

This visually describes how all the parts of the story are brought together to make one story. Another good example is the gif which appears when a person is looking for a story to join.

This shows how the process searches amongst all the existing stories currently on StoryBot and finds the one that has an available slot to join.

The final point was to include an options button which would always allow the user to either leave the current story they are working on, read a new one, view their history, or join a story if they are not writing one at the moment. This was important for it helped ensure that a user never felt lost or unsure of what to do next.

I had included a reminder message for people who were inactive for more than three hours and were also writing a story to prompt them to either continue writing, click “done”, or leave the story. The three hour time interval was quickly criticized as not enough time, so ultimately this ended up being stretched to a day or so. The reminder is also either sent in the morning or around noon in hopes that those are times when a person will have the availability to act.

In a future iteration, I would also want to include a message to ping users who are not writing any story to remind them to start a story and become active, as many just join and never start a story. Furthermore, more time could be spent on the website itself to emphasize the refresh to load a new story feature and a better way to bookmark stories.

Chapter 03 - Launch

After iterating with the feedback I decided it would be time to start opening up the bot to more people. Two places came to mind in addition to my own Facebook profile page to post the bot 1) ProductHunt and 2) Reddit. For ProductHunt I figured I do some reading and found a few cool posts such as this one talking about how the ranking system works and when the best times to post are.

I ended posting StoryBot on Wed, June 1st under Tech to immediately be moved to Games. This ended up working out perfectly as even though this required most users to complete a few extra clicks on ProductHunt, StoryBot ended up getting the first place for Games for the day.

On the heels of this positive wave of feedback, I decided to go ahead and post on the subreddit from which I scrapped the prompts (which by the way are completely attributed on each storybot story page). Before posting however, I wanted to ask for permission by the mods as I am not an active redditor and don’t want to upset the hardcore users by coming off as self promoting. Sadly this was the response I received shortly

As such I took my wins and decided to be happy with what I had. This resistance to Facebook was actually echoed amongst a significant set of people I asked to try out the bot. While my initial intuition was that going with Facebook would make it easier to get more people playing the game, in fact it proved to be an automatic no for some just because of the Facebook association.

Epilogue - 2 weeks later

As of now, here are the Google Analytics for thestorybot.com

With around 100 users, as defined by people who have initiated a conversation with the bot, 32 completed stories, 70 Facebook Page likes, and the Top Game on ProductHunt for June 1st, I would say that StoryBot exceeded my expectations and has left me very happy.

Is this a business? No, I would find it hard to say this is a bot company, but at the same time I am impressed by the general feedback and content produced by the people who participated.

I think that it is too soon to start picking bot winners and losers in terms of companies and ideas as the biggest takeaway from this experiment I found was that most non-tech people just are not aware of bots and their existence. There is very little discoverability for bot services and platforms targeted at normal people, and things that try to make bots more manageable like buttons and simple UI (see below), actually turn out to confuse people at first as these elements have never been used in a chat medium.

I am definitely interested in seeing how companies address and define the use cases, design principles, and expectations of bot-based services in the future.