Authentication and Hosting on SAP Cloud Platform

Did you ever wonder how you can take your bot to the next level and make it more efficient as well as host your own backend in the SAP Cloud Platform?

This article is here to solve this and show you advanced bot building concepts with a sample use case. To that, this article will show you what to do when you have to write your own API as well as how to host it in the SAP Cloud Platform with Cloud Foundry.

Learning goals

1. Get to know a good use case and how it was developed.

2. Learn advanced bot building concepts:

Storing generic user input

A pattern for authentication

Responses with Memory Variables

Custom entities

Bot webhook base URL

3. Understand a working python backend

4. Learn how to host a bot on SCP

Example: this bot solves the problem of forgetting to respond to people who messaged you on a messenger channel.

Prerequisites

In order to get the most out of this article you should have a fundamental understanding of building a bot.

If you haven’t built a bot yet you should have a look at this awesome tutorial on how to build your first bot with SAP Conversational AI.

Check out the resources that we will use for this article:

The bot created on SAP Conversational AI.

The backend code on GitHub.

The Use Case

Problem

Do you also have the problem that you forget to respond to people who write to you on different messengers like Telegram or Facebook Messenger?

Some of your options to fix this are:

Pinning a message which is not very useful as Telegram allows only 5 chats whereas Messenger does not allow pinning at all.

Marking it as unread which gets you confused which messages you actually read but have to work on. Both are not satisfying options when people ask things like “Hey Jonas, could you also get me some concert tickets for Tomorrowland once they are available?“.

Solution

Have a chatbot to forward these messages to, with only two clicks the chatbot will store the messages in your to-do list. In this case we will be using dynalist.io (check it out if you haven’t heard about it).

The chatbot will store the message and annotate any other useful information. This includes the channel from which it came from as well as the contact who provided it. Thanks to SAP Conversational AI named entity recognition we can even extract more information and annotate it in a useful way.

SAP Conversational AI can annotate the following gold entities (others will be added in the future):

Persons: The name of the person will be written in the @PersonName notation to make them easy to search for as well as extract the contact from which the message came from and add it in the note section.

Datetime: It recognizes if someone says “Let’s meet tomorrow at 1 PM”. From this, we can extract the exact datetime and add it in parentheses to make it searchable in Dyanlist as well as use the functionality of Dynalist to add notes with a provided datetime automatically to our Google Calendar.

Location: The location is also a recognized gold entity which means that it knows what someone means saying: “Let’s meet in Disneyland”. We get the information to which google maps location this correspond and can automatically link it to google maps in this case Disneyland.

Email: Email addresses can link directly with a ‘mailto’: link which enables us to open our favorite email program with the email as a recipient preset.

Phone: It can recognize a phone number and link it to a ‘tel’: link which enables us to call the person with one click. Moreover it can format the number in a more readable format and even get the country from a international number. The correct emoji flag will be added in front of the number like +12025550159 –> 🇺🇸+1 202–555–0159

URL: URLs are typically very long. It can recognize them and just display the domain name and link it to the provided URL which makes the note shorter and easier to read.

Emoji: It can recognize text emojis and replace them with real emojis to make the note more colorful like 😀 –> 😃.

Units like mass, distance, and temperature: It can recognize when someone writes “twelve degrees fahrenheit” and replace this with the scalar value as well as add the metric representation 12° F/‘-11.1° C’.

The chatbot will also store the time and the channel it comes from as a note to the message. In the future there will be the a functionality to specify the contact you got the message from.! inbox_bot_setup.gif

My Approach to Solve the Problem

I decided to solve the problem by following these 5 steps below.

1. Define the core function to create the MVP and think about a conversation flow

For example:

#bot [says]: What do you want to do?
#human [responds with Quick reply] "Store a message"

2. Brainstorm other functionalities

3.Implement MVP

4.Test and iterate

5.Polish

Building the bot

Now, after reading about the use case. Let us take a closer look at how it is solved in the bot builder and then see how advanced patterns are applied within this example.

For this let’s have a look on a normal conversation flow and which skills/intents are triggered:

Intents are marked by an @ and skills by a $ whereas other actions are in []. The meaning of @greetings –> $greetings is: @greetings triggers/activates $greetings.

User: Hey!
--> @greetings

Bot: Hi there! @greetings --> $greetings

Bot: Hey this is the dynalist inbox bot. I am here to save your messages to your dynalist inbox.$greetings --> $overview to introduce the bot and show its abilities

User: I want to store a message
[Click Store a message] --> @store-message-inbox

Bot: Hey I need your token to connect to your inbox on dynalist 🙂 Bot: Authenticate with dynalist.io
Bot: On the website you can get your token or generate a new one Bot: https://dynalist.io/developer
[ @store-message-inbox --> $auth-start ]
[stores the current intent as a memory variable]
[ $auth-start --> $auth-ask ]

Bot: Please give me the token 🙏
[ $auth-ask checks if the authentication token for dynalist is provided] [asks the user as it is not yet provided]

User: Hey this is my token IrojuqxabffTM7K30BzBxNfCOD8C__8SwZ_uSNgPTmdPwU6t2_e15-Fm0QT2IRNJ222ONOsu4RG8YbWuMV_VfrpyATP_FFFxWUBVjhvjp_g3mSPR9Novdk0BJMo0Z8B_
[User looks up the token from the provided website] --> @authenticate

Bot: So the token is "IrojuqxabffTM7K30BzBxNfCOD8C__8SwZ_uSNgPTmdPwU6t2_e15-Fm0QT2IRNJ222ONOsu4RG8YbWuMV_VfrpyATP_FFFxWUBVjhvjp_g3mSPR9Novdk0BJMo0Z8B_" ?
[extracts the token from user response]
[repeats the token back]
[ $auth-ask --> $auth-validation and waits for the user input]

User: Yes
[Clicks Yes] --> @yes

Bot: Awesome💪 it works!
[call to the check_token API backend] to verify the token with dynalist
[ $auth-validation --> $get-channel ] to put the right channel as a note (we don't get the information from the conversation so we have to ask the user)

Bot: Sorry can you help me? Which channel is this?

User: This is the webchat
[clicks the Button Webchat from the quick reply]

Bot: Perfect I will keep the channel webchat as a note in your dynalist items 🤓
[storing the channel in memory]

Bot: Please input your message:
[checks if a stored user_intent is in memory]
[redirects to the stored user_intent which is the skill which is names the same $store-message-inbox ]
[checks if the token is present]
[set memory field get_message to true] [ --> $save-message and wait for input]

User: Hey Jonas, could you also get me some concert tickets for Tomorrowlands once they are available?
Bot: Niiiice saved 😎
[checks if variable get_message is present]
[as it is it makes an API call to the end post_to_inbox] this sends the current conversation state to the backend which extracts the text and puts together the call for the dynalist API and posts it to dynalist. It checks the response from dynalist and updates the conversation memory
[as the status code is Ok it post this acknowledgement]
[--> $quick-help ]

Bot: What's next?
[--> $quick-help ] it proposes the next options to quickly access the most relevant commands

Now we know how a user communicates with the chatbot and what happens in the background, let us explore the underlying advanced concepts.

Storing generic user input

We saw in Bot: Please input your message: how we can capture a user’s input. To do this we need two skills:

In the first skill we prompt the user to say something, set a memory variable ‘get_message’ and redirect to a second skill and wait for user input.

In the second skill we check if the memory variable is set and can either send the response to a API (like in this case) or store the plain text in another variable:

After this we just have to clear the first variable ‘get_message’

Authentication Pattern

This pattern is used to ensure that a user can only perform certain actions before being authenticated and provide a standardized three step process to authenticate a user. For this procedure we will also use best practices like remembering an intent, redirecting to a generic skill, validate authentication with a API as well as use yes and no intents for confirmation.

The goal of the pattern is to authenticate a user in an easy way and be easily extendable to be activated before a skill which requires authentication can be used. To that it is scalable to add more skills without major changes which need authentication.

To understand how the pattern work let us look how it is structured. We have three skills $auth-start , $auth-ask and $auth-validation

We can see additionally the following best practices:

Remember Intent: if we have an intermediate skill like authentication which needs to run before another one we can just store the intent in memory to redirect later to it again. We can see this in the example of $auth-start.

Redirect to generic skill: if we want, we can use the redirection method to also redirect to the name of a skill which we stored in memory. This is handy when storing an intent that we catched before and it is named the same as the skill in which it is used.

Validate authentication via webhook: we can do this once we have the access data from a user already validate it by sending it to our API and set a memory flag when sending back the response which indicates if the data was valid which we saw in $auth-validation.

Using yes and no intents for confirmation: when we ask a user to confirm something they can say it in a variety of ways which is why we don’t look if they said ‘yes’ or ‘no’ but rather use our intent system for that. Depending on which intent was triggered we can deduce from the memory variable slug if the user said something similar like yes or no.

Responses with Memory Variables

If you haven’t seen it yet we can access the whole conversation JSON which includes also the memory in responses with double parentheses like ‘{{memory.token}}’ or ‘{{nlp.source}}’.

Custom entities

To create a custom entity, if you don’t have any yet, you go to the intent in which you want to recognize it. Then you write an expression which contains an example of the entity. Now just mark the custom entity with the mouse and you will be given the option to add a new custom entity. The custom entities will be shown on the Train tap underneath the intents.

Bot webhook base URL

This URL is there to facilitate the webhook calls within the bot builder.

When you provide the base URL like in this case: ‘https://dynalistinboxchatbot.cfapps.us30.hana.ondemand.com’ you just have to provide the route to the API when calling a webhook like ‘/check_token’ which will then send the request to ‘https://dynalistinboxchatbot.cfapps.us30.hana.ondemand.com/check_token’.

Like this if the domain of the webserver changes you just have to change it at one point.Python backend

Now that we got to know some of the best practices in bot building, let us dive into how a backend could look like.

We will see how a backend is coded in Python to perform the following:

1. Process a requests

We create a server.py in which we define the different API routes with flask. Each API route should correspond to one distinct function.

It is important to put the host address to 0.0.0.0 to be able to reach the server from outside when deployed on the cloud.

For each route, we define which API from Dynalist should be accessed to the function ‘process_request’.

In here, we extract the JSON of the request as well as instantiate a ‘class RecastConversation’ from the JSON which encapsulates all necessary information from the conversations.

2. Build request and submit API call

We have all the relevant information from the conversation so we can build a response for the API and request it.

This happens in the dynalist_client in which depending on the method called (call_check_token_api or call_inbox_add_api) the right API class is chosen which encapsulates the API address as well as how to build a request for the API. In this way we can easily add new API endpoints from Dynalist without major changes in the code.

Note: We encapsulate the necessary information in the class `DynalistItem` which also takes care of the proper processing of the named entities that were found, as well as constructs a note with meta information.

In the last step the API is called in_call_api and returns the response from Dynalist.

3. Respond to our bot

In the last step, we respond to our bot with the new information we got and we add it to the conversation memory. In this case, we do it in the function build_response which utilized the recast_conversation and the ‘response’ from Dynalist. We need still the ‘recast_conversation’ as the conversation memory we send back is the one that will be used for the future and replaces the old conversation memory.

We send the new conversation memory back which now also contains a ‘status_code’ and a ‘status_message’ that we got from the response from Dynalist and can be used in the bot builder.

Hosting on SAP Cloud Platform with Cloud Foundry

After constructing a full bot, the last step is to deploy it so that we can use it productively.

For this I suggest two options: Development: use ngrok to quickly test all changes and see error messages on your local console.

Download ngrok

Launch: ngrok http 5000

Copy the url ngrok outputs (take the https URL)

Paste it in the Recast.AI interface: Go to your bot page, click on the RUN tab and edit your current bot webhook

Chat with your bot on the channels you’ve configured 😉

Production: deploy it on the cloud. A good way to do this is to use SAP Cloud Platform (SCP) with Cloudfoundry.

In this section we want to concentrate on the deployment on SCP. In order to do this there are three simple steps:

1. Creating an account and configure account

Visit the website from SAP Cloud Foundry

Then click on Free Trial and fill out the data (If you have an SAP account the authentication should work automatically).

Then the space for your trial account should be automatically developed

Next, follow this tutorial to for finishing the account setup

2. Setting up the development environment

After setting up our account we have to make our development setup ready for deployment. This is very well described in this tutorial. You can see the file structure on GitHub. Here is how to navigate to your application

3. Deploying version to SCP

Last just deploy the setup to the SCP with the commands cf push PROJECTNAME or in this case cf push dynalist_inbox_chatbot.

Conclusion

To wrap up we learned about a real use case from idea to deployment and saw different advanced bot patterns involved.

In the process to setup this demo I also learned a lot which I would like to share with you.

It is very important to find a real problem and a good solution first as everything is built around that core solution.

Focus on a Minimal Viable Product (MVP) first to make sure that the solution is solving the problem effectively. To give you an example of how I did this wrong: in my MVP I asked the user for the Dynalist token even though it was not the core functionality of storing messages in Dynalist. The best way here would just be to hardcode the token in the backend. In addition, I realized later that I didn’t even had to code a backend but could have just sent a request directly to Dynalist from the bot builder by forming a custom payload.

Refactoring the backend to an object oriented approach can take some time but can greatly enhance the understandability and extendibility.

Getting a good conversation flow took longer than I expected. For this it also helps to test it out beforehand and mock the conversation flow.

Sometimes you need a lot of training data to recognize correctly a custom entity.

Some messenger reacts differently than others. As an example, in the webchat you can add a link in a list which is not supported in Telegram. You will have to test the solution with different messengers to make sure it works right.

If this article made you learn something new or helped you, help us learn something too and give us feedback by posting a comment or sharing with your friends!