In this blog I’ll be trying to explain in detail the types of chatbots, their development, and how they’re made.
Before we start properly, let’s get a few basics down pat. A chatbot (Conversational AI) is an automated program that simulates human conversation through text messages, voice chats, or both. It learns to do that based on a lot of inputs, and Natural Language Processing (NLP).
Chatbots today are integral in many industries and companies for Social Media, E-Commerce, Customer Services and even Healthcare. Some popular examples are Microsoft’s Zo, IBM Watson or Rasa (A tool used to build chatbots for commercial use).
Let’s get to the real meat then đ¤
Table of Contents
- Types of Chatbots (Rule-based, Intent-based, Open-Domain)
- Chatbot Architecture
- Natural Language Processing
- Industry Examples
- Further Research and References
Types of Chatbots
Rule-Based Chatbots
Rule-based chatbots are also referred to as decision-tree bots. As the name suggests, they use a series of defined rules. These rules are the basis for the types of problems the chatbot is familiar with and can deliver solutions for.
Like a flowchart, rule-based chatbots map out conversations. They do this in anticipation of what a customer might ask, and how the chatbot should respond.
Rule-based chatbots can use very simple or complicated rules. They can’t, however, answer any questions outside of the defined rules. These chatbots do not learn through interactions. Also, they only perform and work with the scenarios you train them for. Opting for his type of chatbot comes with several benefits, like:
- The chatbot doesn’t need extensive training which makes the implementation process faster and less complicated.
- By pre-defining the structures and answers, you can better control the behavior and responses of the chatbot.
However, rule-based chatbots have their limitations, with a few of their disadvantages being:
- A rule-based chatbot can’t capture typos which means that in some cases it won’t understand the guest, which can cause frustrations.
- The interactions with a simple chatbot feel robotic rather than conversational.
- They cannot learn on their own which means that any improvements need to be made manually.
Trending Bot Articles:
2. Automated vs Live Chats: What will the Future of Customer Service Look Like?
4. Chatbot Vs. Intelligent Virtual AssistantâââWhatâs the difference & Why Care?
Intent-Based Chatbots
In comparison, AI chatbots that use machine learning understand the context and intent of a question before formulating a response.
An AI chatbot is a chatbot powered by Natural Language Processing. So, unlike with a rule-based chatbot, it won’t use keywords to answer, but it will try to understand the intent of the guest, meaning what is it that the guest wants. The more it interacts with guests, the better it will become at understanding the intent, and the better it will become at answering guest requests.
The way they work is that they capture the intent behind a question or response. For example, a user would ask something like âHello! How are you doing?’ or even something like âHeyoooo’. Despite these being wildly different, the bot captures the intent as `greeting` so it responds with a comment corresponding to that.
Open-Domain Chatbots
Arguably, Intent-Based chatbots are also AI-Based Bots. These chatbots operate on a semi or completely Artificial Intelligence backed algorithm. A bot built on GPT-3 would be a perfect example of this, where it understands the context and responds completely on it’s own.
This type of bot is much more suited for complex kinds and a larger number of queries. Choosing an AI chatbot comes with several advantages, like:
The chatbot can understand typos and grammatical mistakes, so it is still able to respond to the question.
It will continuously improve without much further assistance.
Conversing with an AI chatbot feels much more natural and human-like.
Although Artificial Intelligence is an advanced technology, this bot also has its limitations:
- AI chatbots go through a learning process, which makes their implementation process more complicated and longer.
- By not using pre-defined structures, the conversations led by an AI chatbot are less predictable.
- When a chatbot was wrongly taught something, it takes a while before it âunlearnsâ and can learn the correct behavior.
Chatbot Architecture
NLP Engine
The Engine is the core component that interprets what users say at any given time and converts the language to structured inputs that the system can further process. Even if a chatbot is doamin specific, it needs to contain and harness a vast amount of information. The NLP Engine helps with that.
It contains advanced machine learning algorithms to identify the user’s intent and further matches them to the list of available intents the bot supports. It can be understood to have two components:
- Intent Classifier: Intent classifier takes user’s input identifies its meaning and relates back to one of the intents that the chatbot supports.
- Entity Extractor: Entity extractor is what extracts key information from the user’s query.
Knowledge Base
This is the key component in answering users’ frequently asked questions. Q&A systems interpret the question and respond with relevant answers from the knowledge base. It can be trained manually or trained to do so itself
- Manual Training involves the domain expert creating the list of frequently asked users queries and map its answers. This helps the bot quickly identify the answers to the most important questions.
- Automated Training involves submitting the company’s documents like policy documents and other Q&A type of documents to the bot and asking it to train itself. The engine comes up with a list of questions and answers from these documents. The bot then can answer with confidence.
Data Storage
The data storage is simply a store of previous interactions to feed to the nlp engine so that the bot retains a semblance of context during a chat. This is especially critical to remember knowledge about the user for further interactions.
Natural Language Processing
Natural Language Processing in Chatbots finds a way to convert the user’s speech or text into structured data. Which is then utilized to choose a relevant answer. Natural Language Processing includes the following steps;
- Tokenization: The NLP separates a series of words into tokens or pieces that are linguistically representative, with a different value in the application.
- Sentiment Analysis: It will study and learn the user’s experience, and transfer the inquiry to a human when necessary
- Normalization: This program model processes the text to find out the typographical errors and common spelling mistakes that might alter the intended meaning of the user’s request.
- Named Entity Recognition: The program model of chatbot looks for different categories of words, similar to the name of the particular product, the user’s address or name, whichever information is required.
- Dependency Parsing: The Chatbot searches for the subjects, verbs, objects, common phrases and nouns in the user’s text to discover related phrases that what users want to convey.
Industry Examples
Meena by Google
Meena is an end-to-end, neural conversational model that learns to respond sensibly to a given conversational context. The training objective is to minimize perplexity, the uncertainty of predicting the next token (in this case, the next word in a conversation).
At its heart lies the Evolved Transformer seq2seq architecture, a Transformer architecture discovered by evolutionary neural architecture search to improve perplexity. Find out more about it in Google’s AIÂ blog.
Replika
Replika was founded by Eugenia Kuyda with the idea to create a personal AI that would help you express and witness yourself by offering a helpful conversation. It’s a space where you can safely share your thoughts, feelings, beliefs, experiences, memories, dreamsâââyour âprivate perceptual world.â
Essentially, Replika is a chatbot that while talking to you, gradually learns to imitate you till it becomes you.
Tay by Microsoft
Tay was an artificial intelligence chatter bot that was originally released by Microsoft Corporation via Twitter on March 23, 2016; it caused subsequent controversy when the bot began to post inflammatory and offensive tweets through its Twitter account, causing Microsoft to shut down the service only 16 hours after its launch. It has since been seen as a case study on how user interaction can ruin chatbots.
Further Research and References
For the purpose of creating this blog, we referred to several sources which can be found below. Apart from those, if you are interested in learning about or developing chatbots, we’d encourage you to check out Rasa, a popular open-source library used to develop chatbots. And there’s also the Chatbotslife Magazine right here on Medium!
And here are our sources:
- âTowards a Conversational Agent that Can Chat AboutâŚAnythingâ, Google Research: Brain Team, 2020
- âUnderstanding the Architecture of Conversational Chatbotsâ, VSoftConsulting Blog, 2019
- âThe Rise of Social Botsâ, Ferrara Emilio, Varol Onur, Davis Clayton, Communications of the ACM, July 2016
- âWhy Microsoft’s âTay’ AI bot went wrongâ, Hope Reese, Tech Republic, March 2016
Donât forget to give us your đ !
Machine Learning in Chatbot Development was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.