Building a chatbot with Rasa

You visit a website, you see a small chat icon on the bottom right corner(most of the times), you click on it and it starts with a greet like a human(sometimes even humans will not greet you in the way it does). It has the answers to all the questions you have related to the context and answers are quite accurate. Well, ‘it’ in this context is a Chatbot/AI-assistants developed to recognise the intent, find the entity and respond the search results with seamless dialogue flow. This article will give you a brief about developing a similar application using a popular framework.

Image source: www.ecommerce-nation.com

Frameworks make the life of a developer much easy. We have some for building a website, to convert a website to a desktop app and yes also to build a complex chatbot. With a basic knowledge of natural language processing, it becomes very easy to build a simple chatbot specific to a problem statement. By this I mean, unlike a google assistant or Siri or any other digital assistant, the chatbot will be developed only to answer queries specific to the context for an example, a movie ticket booking chatbot will not give any answer to ‘How to lose weight?’. Rather they will be build to answer only the domain-related queries. We can build a generalised chatbot but then it will be computationally expensive.

Rasa framework

Rasa is basically an open-source conversation AI framework using which one can develop and integrate an AI-assistant in a web application or also we can deploy it on platforms like Slack.

For Rasa installation, you can refer clean and simple documentation provided by the Rasa developers or else refer to some youtube videos. After successfully installing the required things and downloading/cloning Rasa, we will be provided with multiple files around which we will be building the bot. Before diving into the files lets first see how Rasa works.

Rasa is divided into two parts -

1. Rasa NLU
2. Rasa Core

Rasa NLU

The first part involves natural language understanding where we classify the input user query into one of the intents and also extract the entity present in it. To get what exactly intent and entity is, let us consider an example. Assume that we are building a restaurant search chatbot and the user inputs— “Find a good restaurant in Pune which is famous for Chinese food”. Now, the intent in the query is restaurant_search and the entities present in the query are Pune, Chinese food. So Rasa finds the intent and entities from the query in the NLU part and stores it into respective variables which will be provided to the core in a form of JSON.

{"intent":"restaurant_search",
"entity": 
        {"entity_value":"Pune",
         "entity_type":"location"},
        {"entity_value":"Chinese",
         "entity_type":"cuisine"}
}

This helps us to identify the exact meaning of each query and then extract the important words that will be used to fetch the exact details.

Rasa core

It is the brain of the framework. In here, we specify/train the whole dialogue flow that the bot can take and we can generalise by specifying the flow examples which are also called as stories. We also need to add the backend logic in this section. Logic will involve API calls to get the details and send them as responses to the user. Other things like if a mail need to be sent, we can write a function for the same. The extracted entities from the NLU needs to be used here as query parameters for the APIs.

Coming back to the files in the Rasa framework, we get a bunch of generated files amongst them there are four important files —

1. nlu.md
2. stories.md
3. domain.yml
4. actions.py

In order to extract the intents and entities correctly, we need to have some examples so that the NLU can be trained on those and perform accurately on test data. We also need to specify the intent to which the example os referring to. Example-

## intent:restaurant_search
- Find a good restaurant in [Pune](location) which is famous for [Chinese](cuisine) food

This will tell our bot what different intents it needs to look for. ‘domain.yml’ is a kind of declaration file, where we declare all the different entities, intents, actions, responses that the bot will utter and the other data storing variables which are called ‘slots’. In ‘stories.md’ the file we need to specify some dialogue flows that user can go through. This creates a probability map for all the actions that bot can do next and select one with the highest probability. Example story-

## story_1
* greet
- utter_greet
* restaurant_search{"cuisine": "chinese", "location": "Pune"}
- slot{"cuisine": "chinese"}
- slot{"location": "Pune"}
- action_search_restaurants
* affirm
- utter_goodbye
- action_restart

Notice the ‘*’ and ‘-’ in the story. User input is denoted with ‘*’ and the bot response/action is denoted by ‘-’. Stories can be seen as different combinations of the flows which make the chatbot even more flexible.

The actions.py is a python logic containing dedicated functions for each action to perform various backend tasks. Here we can get, modify and utter the response by using python and it can be seen as a server that we must keep running throughout the flow.

Rasa NLU and Rasa core can be trained separately or at the same time and we can check the performance of both using the CLI commands provided by Rasa. Another important feature that Rasa provides is ‘Interactive learning’. After executing this command in CLI, we can see the whole process of intent classification, entity recognition and action flows. Also, if the next action is wrong, we can correct it at the particular instance and the chatbot learns from the changes we do. Finally, after checking the entire story, we can then merge this story into the stories.md file so that next time bot will be trained on it. This was one of the most amazing features that I came across and hats off to the developers. We get the trained model into the model’s folder which we can use to integrate and deploy.

It’s quite interesting to see these frameworks in action and I was very amazed by just thinking about the development involved in developing Rasa or other similar frameworks like Google dialogue flows. But definitely, to make core algorithmic changes it is always preferred to come up with our own natural language processing steps to customise the final model according to our problem statement. Thank you for reading :)