How to build a Twitter Search Bot using Microsoft Bot Framework and Cognitive Services

Introduction

Last week at its annual developer conference, Build, Microsoft announced the new Bot Framework in the attempt to get developers to build intelligent bots using Microsoft technologies.

In this article, we will introduce the concepts of Conversational AI and bots, and will create a bot using the Microsoft Bot Framework that will search Twitter for tweets containing the user query.

We will then integrate it with LUIS (Language Understanding Intelligent Service) from the new Microsoft Cognitive Services which will allow users to input natural language. Then, with the help of Machine Learning, we will extract the intent from the user’s query and search Twitter.

What is Conversational AI?

The Conversational AI is not a new concept. I was first introduced to the idea of a conversational bot back in the early 2000s when I was playing with ELIZA, an early example of how to use natural language processing to achieve a very near human-like conversational bot.

The idea behind conversational AI is to have a computer program respond to the user as close as possible to a real conversation.

While early programs like ELIZA were based on predefined scripts, the newer ones are based on Artificial Intelligence and Machine Learning.

(Photo source here)
ELIZA is a computer program and an early example of primitive natural language processing. ELIZA operated by processing user’s responses to scripts, the most famous of which was DOCTOR, a simulation of a Rogerian psychotherapist.

Using almost no information about human thought or emotion, DOCTOR sometimes provided a startlingly human-like interaction. ELIZA was written at the MIT Artificial Intelligence Laboratory by Joseph Weizenbaum between 1964 and 1966.

After having read a lot of Medium and TechCrunch articles in the past months about how bots are the new apps (here, here or here), it became pretty clear that the next step in the user interaction is represented by intelligent bots.

What is a bot?

A bot is a piece of software designed to automate a specific task. When talked about in the context of conversation as a platform, a bot becomes the chat interface of a regular app.

So you should allow tasks that required full UI to be performed by the user only through conversation.

You just send and receive messages from the bot. No need to learn, understand and navigate disparate interfaces or languages. Users will be able to interact with bots just as they interact with other humans. It’s the most natural way to communicate and transact.

The goal is to have the user input natural language and your bot to perfectly understand and execute the action the user wants.

Of course, you can do this by using terminal-like commands (much like inviting a user to a Slack channel), but then you compel the user into remembering commands and arguments (then, every bot will have specific commands and arguments).

Microsoft Bot Framework

Build and connect intelligent bots to interact with your users naturally wherever they are, from text/sms to Skype, Slack, Office 365 mail and other popular services.

Searching Twitter

After you create it, you will be able to find the consumer key, consumer secret, the access token access token secret that we will use in order to authenticate the calls to the Twitter API.

We will store the tokens Twitter generated for our app in order to use them when searching. Since we don’t want credentials in a Git repository, I am created a file called appSettings.config in which I placed the tokens from Twitter and the authentication tokens from LUIS (we will see this later).

We are going to use a library called Linq2Twitter in order to make Twitter queries easier, and we will install int from NuGet.

Creating the Twitter Client

At this point, we are ready to create the class that will be responsible for actually executing Twitter searches. At first, we will only execute un-intellingent queries, meaning that we will search Twitter exactly for the input from the user.

As I said, we are implementing a wrapper over Linq2Twitter that is simply going to get tweets based on the user query.

As we can see, we simply create the reply message using the string that contains the tweets received from the search.

I limited the results to 3 tweets because the message was rather large. You can have up to 200 tweets using the Search API. For more tweets, consider using the Streaming API.

When running the application and testing it with the Bot Framework Emulator, if we input Satya Nadella, we can see 3 tweets that satisfy the query:

Now let’s try to input some natural language and see how the system behaves: what does the world think about bill gates?

Let’s see what happened here: in the first query, the input was simply: satya nadella, so the TwitterClient just made a query for satya nadella.

The second time, the TwitterClient did the same thing, this time with a much more complicated query: what does the world think about bill gates?, query that returned no results from Twitter.

Adding support for Natural Language Queries – LUIS

So far, we managed to create a bot that receives queries and makes Twitter searches based on the exact text of the query.

The goal is to have the user input natural language and for the system to extract the intention of the user, as well as the entities of the intention.

For example if a user had the following input: what does the world think about bill gates?, the system must understand that the user wants to do a Twitter search for bill gates.

For the following input: how is apple doing?, it should perform a Twitter search on apple.

One of the key problems in human-computer interactions is the ability of the computer to understand what a person wants, and to find the pieces of information that are relevant to their intent. For example, in a news-browsing app, you might say “Get news about virtual reality companies”, in which case there is the intention to “FindNews”, and “virtual reality companies” is the topic.

LUIS is designed to enable you to very quickly deploy an HTTP endpoint that will take the sentences you send it, and interpret them in terms of the intention they convey, and the key entities like “virtual reality companies” that are present.

LUIS lets you custom design the set of intentions and entities that are relevant to the application, and then guides you through the process of building a language understanding system.

Once your application is deployed and traffic starts to flow into the system, LUIS uses active learning to improve itself. In the active learning process, LUIS identifies the interactions that it is relatively unsure of, and asks you to label them according to intent and entities.

This class is responsible for retrieving the intent and entities from LUIS based on a query. As you can see, the GetLuisResponse method accepts a message, then makes an HTTP request to your app’s endpoint, then formats the result as LuisResponse.

In the MessagesController, we will modify the result so it uses the intent from LUIS.

Right now, we can add more stuff to our bot: we can add confirmation, remember the last query, build more complex queries, add more intents and entities and check them, stuff we will make in part 2 of this article.

Bot Conclusion

We started by building a pretty straightforward bot: the user would put some exact phrase, then our bot would search Twitter for that phrase.

Then, we added support for natural language queries, meaning the user could input rather complex phrases, then, using LUIS, we would extract the intent and the entities from the phrase.

Conclusion

You just send and receive messages from the bot. No need to learn, understand and navigate disparate interfaces or languages. Users will be able to interact with bots just as they interact with other humans. It’s the most natural way to communicate and transact.

This phrase basically states exactly what a bot should be all about, and bots are becoming the new apps.

This way, we can build bots for the web, for Skype, Slack, SMS and many others.

For a step-by-step tutorial on how to deploy your bot and use it with the Bot Connector, check this resource.