Month: July 2022
Before you’re ready to have automated conversations with your customers at scale, you should ask yourself — how exactly do you take your idea and turn it into a real chatbot?
Here are the 10 steps you should take to go from idea to working chatbot prototype that’s ready to be built.https://medium.com/media/6bb664ab31de248fdcefe61c73489c90/href
One question that comes up a lot when a business or organization wants to create a chatbot is how exactly do you take your idea and turn it into a real chatbot? Before you’re ready to have automated conversations with your customers at scale, there’s a process of strategy, conversation design, and testing that needs to happen. Here are the 10 steps you should take to go from idea to working chatbot prototype that’s ready to be built.
- Define the purpose. This is the most important thing to determine for your chatbot. Why are you creating a bot in the first place? If you are taking an existing interaction or process and automating it, what is that current experience like, and how could a bot help the customer or improve the process?
- Define the goal. Now that you understand the why, what will this bot do? What is the valuable outcome that a user will gain by interacting with it? It is essential to define your chatbot strategy before the writing begins, so you know exactly what the bot will do and why that goal is important.
- Outline the steps. With that end goal, work backward to determine all of the steps necessary to reach that goal. You can likely gain the information from the business website, sales materials, interviews with customers or sales agents, etc.
- Define the audience and personality. In order to design an experience that converts, it’s crucial to know what the user wants and what their sentiment will be during the interaction. To connect with the audience, you have to know them! How do they buy? What are their challenges? How familiar are they with your topics? These are all questions you will need to answer.
- Map the flows. Create a visual guide of your steps, and fill in the ways these connect to each other. The main goal with creating a flow map is to visualize how a user would go from entry to exit and where they might want to — or be able to — cross paths into other flows. There are several tools you can use to easily create a flow map that represents your chatbot user journey. My two favorites are draw.io and Lucidchart.
- Write the key flows (Hello, Main, Outcome). With these conversations design best practices in mind, it’s time to write! Begin with the most necessary parts of your chatbot conversation, and write the beginning-to-end “ideal” experience. This will also help you discover offshoot flows you need to add. I’ve created an online template for writing all of the dialogue for your chatbot, including these flows, but you may want to write yours in another internal tool or document, depending on your specific needs.
- Write the secondary flows (other answers to main questions, about, contact, etc.). When writing the main flows (and in your flow map), you will notice there are many points where flow needs to extend to offer another path, if a user selects a different answer. After you create the main flows, go back and fill in anything that is currently a dead end and make sure all of your flows connect.
- Create a demo/prototype. Turn your 2D writing into a working, 3D conversation. Creating a video mockup or working example of a chatbot is the best way to illustrate the experience and demonstrate it for your team. Use a tool, like Botmock or Botsociety, to easily create a working demo so you can see how the conversation really flows.
- Edit the experience. After using your prototype and sharing it with your team, you’ve likely uncovered messages that are too long, along with dead ends or pathways that don’t make sense. As necessary, go back and edit the copy to make sure it’s as effective as possible.
- Test the experience. You can continue to test the experience in a prototype mode or in a production-ready bot. Repeat this as necessary. There are a few different ways to test a chatbot (or a prototype) before you release it out into the world. There is role-playing, usability testing, or getting user feedback from other chatbot professionals online. Once you have an experience you are happy with, it’s a good idea to test it with a small group of customers and scale up to something that’s available for everyone to use.
Once you have completed these steps, you are ready to build and release a chatbot. You should feel confident that the experience will be enjoyable for the customers and help the business reach its goals.
Feel a little bit overwhelmed by the process? Not sure where to start? Want to go even further? Learn the basics of conversation design, the ins and outs of this exact process and create a prototype when you take my online course, Chatbot Writing & Design.
I’m Hillary, the Head of Marketing & Conversation Design at Mav.
Want to chat bots? Network with 1500+ conversation designers when you Join my Private Facebook Group!
I work at an IT services company (Simpragma) which aims to revolutionize Contact Centres with their automation expertise built over a decade. The company has built Voicebots, chatbots, social media messenger bots, visual IVR, etc for renowned brands where I have contributed as a user experience designer.
This project was done for a leading micro-finance company (we will use the name “B Finance”) which has over 5 million customers. They wanted to automate payment reminder calls as the task is repetitive in most cases. I would take you through the design process of the Collections Voicebot at a glance where some stats might be altered for confidentiality. As we had prior knowledge of B Finance’s customer base and had developed a voice bot for them earlier this project was more like a feature roll out to a small segment of customers first, test and improve for final launch. Specific things to keep in mind while going through any Ux case study.
- UX maturity of the design team at Simpragma.
- Duration for research and Ideation –2 weeks.
- Business impact on B Finance in terms of expenditure and ROI.
- Transition for B Finance’s customers from receiving calls from human customer service executive to an AI-ML-based voice bot.
How does the Voice bot function
The voice bot uses ASR (automatic speech recognition) to convert speech-to-text ANd NLP (natural language processing) to understand human language. Then the response is generated using TTS (text-to-speech). The AI predicts the bot response from a predefined library and Machine learning helps to train the voice bot over time for complex scenarios.
For more information on the voice bot process, click on the links below:
- What is Voice bot: An Ultimate Guide for Voicebot AI
- Voicebots 101 Guide: applications, benefits, best practices – SentiOne
Earlier Project with B Finance -Inbound customer query resolution Automation
This was not a completely new project from scratch. Our team automated the inbound customer service calls for B Finance earlier and provides continuous support for upgrading workflows, scripts, intents, and any other issues. B Finance was able to answer approximately 5000 calls per day with 50 telephony lines and 22 agents and approximately 3000 calls were going unresolved. After automating the calls B finance scaled up 3 times easily as it did not require hiring and training too many agents. 80 percent of calls were handled by the voice bot and other unique cases would end up with human agents if needed. The voice bot was available 24*7 and 7 days a week as well. The voice bot is continuously trained through machine learning for handling more complex queries over time.
Doing a project with B Finance already gave us a lot of insight into their customer base. Most of the customers have taken small loans for products like mobile phones, home appliances, or other electrical gadgets. As the majority of users were located outside main cities:
- We understood customers’ level of literacy and understanding of technology.
- The majority of the customers speak Hindi in different styles.
- They would use different kinds of words for the same query. So, we were able to develop a library of intents (synonymous words) that can be recognized by the bot to understand the user’s query. For instance, some users might say Due EMI and some might say pending installment but it means the same. The bot could understand similarities and differences as well using such intents.
- A lot of users perceive voice bots as a human who can set higher expectations for getting their issues resolved. We make sure to build a human-like bot but not exactly like human customer care agents.
- As everyone has a different grasping speed, we tuned the bot to speak with an average speech rate, but some customers might still interrupt it. The voice bot was built to adapt, understand and answer to such customers as well.
- Some customers understood that it is not a real human and thus they demand to speak to one. If the customer is adamant, the voice bot transfers such calls to human agents. It would also transfer in some specified cases as well.
After getting the inbound calls automated, B Finance approached us for automating their outbound collection calls. Like any other Financial institution, B Finance generates revenues from the difference between what it lends and what it receives back in form of EMI (principal + interest). Regular payments ensure a fund balance for B finance and they can lend the money further to its new and existing customers. Thus collecting payments on time and reminding customers to pay is a crucial operation for B Finance.
Customers need to be reminded regularly about upcoming and overdue EMI payments. Especially if a customer hasn’t paid the EMI on time it affects their credit score and they have to pay penalty late payment charges post the grace period.
This project would cover 8 lac customers approximately with 300 calling lines every day.
As the client was already operating payment reminders and due collections manually through human agents, they were able to give us data on different scenarios to be automated and scripts for the voice bot as well
These scripts are not finalized until we develop a minimal viable product for testing and improvements.
Insights from the scripts
Scripts were not just dialogue between a bot and user but also gave what business required customers to be reminded about.
- Regular Payment Reminder– Reminding customers in advance about upcoming EMI payments.
- NACH reminder– NACH is a facility given by banks in case they want to pay directly for EMI from Bank accounts. Sometimes customers forget to update NACH with their banks and have to be reminded before the due date to avoid late payments.
- Pending EMI reminder– In case a customer forgets to pay EMI on the due date, B Finance reminds them to pay to avoid penalties and negative effects on their Credit score.
- NACH update pending/failed– In case NACH did not get approved B Finance cannot debit from the customer’s bank account. Thus they are asked to pay by other methods.
- Smart Debit failed– In case NACH/smart debit was set up but customers’ accounts could not be debited due to low balance, either they can pay by other methods or add funds to the bank account as B Finance will re-attempt again in a few days.
- Advance Reminder Call– A bit overlapping but can be a necessary step for customers with bad payment history.
- Follow-ups– Scripts for follow-up calls were shared too.
- Disconnection– Script might change in case of a call disconnection due to network reasons.
- Customer declines/doesnt answer– Bot might have to be more assertive or might need a change of plan
Quick Secondary Research
In a B2B scenario, it wasn’t easy to have an understanding of a competitor’s product. After spending a couple of hours getting a demo from competitors I finally moved on to secondary research and came up with some crucial steps to follow for our MVP.
- Introduce Yourself Confirm Name/DOB etc (Identify correctly)
- Confirm Name/DOB etc (Identify correctly)
- Check availability of customer (good time to talk)
- Tell the intent of the call
- Empathize with customer
- Help the customer with ease of payment or alternatives if you can
- Inform about consequences like late payment charges etc.
- Inform the customer properly about the next steps
- Send necessary documents if required
- Give a contact number for the customer to call back in case
Initial Conversation Flow
The initial flows were created for an initial conversation with the client. It is easier to have a conversational flow in place to understand voice bot and customer interaction for both the development team and client as well. It acts as a base for information architecture as well where the dev team can design the decision tree based on the flow in the later stage.
Here is a glimpse of one crucial flow of Pending EMI reminders out of the 6 flows that I created for discussion. The other flows have certain similarities as the nature of customer response was similar. It would give you an idea of what I presented to the client.
Some more cases were added along with each flow
- Follow up — in case customer doesn’t pay after reminder call as well. voice bot to call back and explain the consequences of delaying it.
- Network issue– Voice bot will call back after some time and apologize for the inconvenience caused.
- Customer not answering– If the customer responds after some time the voice bot will be more assertive and tell the customer to pay to the matter. In case a customer doesn’t answer at all, the call logs are maintained and the human agent tries to reach them.
For the current release, the phone lines are limited to 300 for Eight lac customers. If an average call takes 4 minutes to be completed and the number of operational hours is limited to 12 as per government guidelines. Voicebot will be able to call 43000 customers per day approximately.
In case, the due date had crossed the flow would be similar with the just intent message being changed. In case of general reminder, the flow would be like this
We had a quick call with the client and went through the flow together. We even did a small exercise of roleplay where one participant was a user and the other became the voice bot. A few important points that we discovered
- Checking availability was time-consuming and did not fall into the bracket of a ‘must do’ or ‘should do’ at this stage so it can be added later. The reminder call was also very crucial for businesses to generate revenue and as a priority for clients, we decided to skip this step.
2. We were updated with information about the mode of payment, customers need not visit a branch to make payments.
3. For any dispute with the dealer or if the customer has already paid etc, a human agent would call back if the voice bot is not able to resolve the issue.
I wanted some more insights from B finance’s customers but due to a time crunch, it wasn’t possible. We decided that I would analyze their reaction and behavior from the call recordings taken after the first release.
I still went ahead and did a quick qualitative study on agents and process manager working in Collections process for various companies.
- What is the collection process that you follow?
- How often do you call the customer?
- Is there a right time to call the customer?
- What is your tone while talking to customers
- What is the strategy to make the customer pay their dues?
- What is the repeat rate of defaulters? Are you able to check the history of late payments for one customer?
- What are the consequences of not paying on time?
- How do you deal with people who do not pay regularly?
- It is not easy to get users to pay as they come around with various excuses.
- Its human nature that agents might get aggressive or rude dealing with similar situations every day. There is pressure for meeting targets as their earnings are largely dependent on incentives.
- Financial institutions have incentive-based income plans for agents to avoid low competency as the revenue depends on EMI payments from customers.
- There is no hypothecation in microfinance loans where the annual income of households is low. The company cannot claim the items back from customers and neither it would serve a greater purpose after re-selling these products. Getting installments is the only way for growing the business.
- Agents might inform customers about the repercussions of having a low CIBIL score, but either user are unaware as they are from low-income groups or they are not bothered with low CIBIL scores as well.
- Agents go out of the way to collect EMI amounts from customers. There is no ideal way to date that can assure timely payments from all customers. Bad debt is a huge problem for financial institutions.
- Not ideal but agents call customers’ relatives and friends to build some pressure on customers as they might pay out of embarrassment. Similarly, they scare customers about recovery agents turning up at their office or residential addresses.
- The idea of embarrassing people does work somehow. It might be a societal thing as well and for that, we need more research which I might take in the next phase.
- The call frequency is higher and more assertive tones are used for habitual offenders.
- People block or avoid calls so agents might use alternative numbers.
Based on the above insights I tried to map down the selection criteria for calling. There were various differentiators and categories to be followed like
- Customers with good payment history vs bad payment history: Voicebot would need to understand this to change the tone with help of a script.
- Pre and post notification: voice bot needs to identify whether it is calling before or after the due date.
- Call Intent: Voicebot needs to inform the customer about the reason for the call like a general reminder for payment, pending EMI, missed auto-debit or NACH failed.
- The intensity of reminders: Customers need to be reminded and the ones who still have not paid might have to be reminded .
- Timeline: Intensity has to increase with timelines. It depends both on business requirements and voice bot availability.
- Identification: The customer needs to be identified properly before going ahead with the call else updating the records.
2nd Draft (Ideation)
So I started building flow according to the selection matrix where the intensity increases every 30 days if a customer doesn’t pay until 60 days and then goes to field recovery teams.
The call frequency also increased with time and in the case of call avoiders, human agents would reach back.
As the voice bot would have a certain limitation, human agents would take over in unique dispute cases, disagreements for payment, payment gateway issues, etc as it was too early for the voice bot to handle it.
While I was creating flows with different intensities of messages over time for defaulters, I did a quick check with my developers. I felt the selection matrix was overwhelming and there were too many flows to be followed for the prototype.
My assumption matched the developers’ thoughts as well and we concluded to mellow it down.
3rd Draft (Ideation)
For the prototype to be developed quickly with minimal effort yet good enough to be tested in a real environment with approximately 150 people I reduced the complicated matrix into a simple one.
After a lot of ideation, I was able to combine all flows except NACH reminder as it did not require the customer to tell for payment agreement or disagreement.
Flow for Development
The combined flow would be used to develop the prototype. We chose some must-do tasks to be taken care of for the prototype as all problems cant be solved at once.
- Voicebot will fetch customer data through an API to determine the due date Due amount and other necessary details.
- The voice bot would confirm if it speaking to the right customer, If not it would try to get the right phone number and record it. Later it would be manually verified and updated by human agents.
- Voicebot would then introduce the intent to the customer after checking the condition like Check Condition
10 days Before the due date-
1 day before the due date
After due date
Smart debit failed
4. The script would get severe with time for customers who do not pay. For the current prototype, we removed the differentiator of good and base payment history as well.
(0–15 days) — Normal severity
(15–30 days)- Medium severity
(30–60days) high severity
5. Voicebot will give clarity of payable amount — Due amount +interest + late payment charges.
6. In case of any dispute where a customer does not agree to pay voice bot will end the call with a message; I.e. customer care agent will call back the customer.
Development of prototype
As we already have prior knowledge and some modules ready for the same client the development for the first release would take 4 to 6 weeks where I work closely with developers to take insights and note down gaps and opportunities.
Scripts and Intents
Scripts are generally improved over time with every release. Similarly, new intents(words told by a customer that the bot can use to understand their query) are continuously added to the library for training the voice bot.
Like every other contact center, we record and audit the calls along with an audit manager from the client side. We listen to 10 calls per day to understand what went right and what did not. Then we divide tasks according to tech issues or process issues and make improvements continuously.
The data we get from calls are also used to train the bot more every day for new intents and scripts for the voice bot are modified as well.
Testing will give us insights for improvements for the next releases but we do have some key factors already to look at.
- Currently, the phone lines are limited to 300 and thus a customer might get a follow-up call only after 18 days. We need to increase phone lines or find a way to prioritize follow-up for regular violaters.
- We need to add and test for certain steps like ‘checking availability with customer ’ which we removed earlier.
- We need to personalize the calls and change scripts accordingly.
- Train bot for complex scenarios to reduce human agent intervention.
- Get qualitative insight from B Finance’s customers.
- In case a customer doesn’t answer should the voice bot use the alternate number to call back or call the customer’s friends and family to reach out?
- Should there be a ground team for collecting EMI offline?
- I will work on convincing people to pay. To date, offline physical recovery has worked best in case of bad debts. Empathetic user research with such customers will yield great insights.
- In case customers block one number what would be an alternate route to reach.
I will add the further progress of this project in the next case study. Stay tuned.
Ux Case study-Automating Outbound Collection calls through Voice-bot for a Micro-Finance Company was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.
It’s 2022 — Time to Level up your Conversational AI Experiences
The Conversational AI (CAI) space has come a long way over the last few years. As users prefer to communicate over digital channels, the demand for conversational AI continues to grow. Organizations worldwide are increasing their CAI investments in response to this trend and maturing how they leverage Conversational AI to supplement customer service agent interactions to deliver seamless customer experiences across a multitude of channels.
Customers’ expectations have also matured due to the proliferation and ubiquity of conversational interfaces and virtual assistants. They now demand easy, effective interactions that are personal and contextual to their current needs.
To truly elevate mundane conversations, improve engagement, and add value to their customers in 2022 and beyond, organizations need to focus more on the following practices to craft delightful experiences.
From FAQs, to transactional experiences
Conversational interface projects often start with a proof of concept involving launching a virtual assistant that can automate responses to frequently asked questions (FAQs) via chat or voice. Organizations that want to increase customer satisfaction and achieve business goals need to start looking beyond just FAQs to reap the actual benefits of conversational AI.
Today’s advanced Conversational AI systems that utilize natural language understanding (NLU) can automate many complex transactions to make life easier for customers and internal teams. For example, banks could enable bill payments via virtual assistants instead of just navigating customers to a ‘how to pay’ webpage. A food retailer could allow customers to order food using a virtual agent rather than just navigating to a ‘menu’ page on their website. Check out more Use Cases of Conversational AI in the Finance industry to increase customer satisfaction and automate your processes.
Building a transactional virtual assistant does not necessarily mean total call center automation with all possible transactions. Organizations need to take a structured approach and use data to prioritize key transactions that are high-volume and high-impact. This will help them deliver more value to their customers and move them closer to meeting their business objectives.
From linear, to flexible and cyclic conversations
Organizations need to be mindful that they are creating experiences for real people who are on the other end of the virtual assistants. Therefore it is paramount to keep customers in mind during the entire process. This shift is profound and places the onus on organizations to deliver a seamless user experience to lessen the user’s cognitive burden.
To continue providing a fluid customer experience, organizations need to anticipate and map out every possible scenario, query, and customer response. They need to design flexible conversations so that customers can converse using their own words in addition to picking from pre-defined menus. They should also be able to change the direction of dialogue or request additional information along the conversation’s path. Lastly, the conversation design needs to be cyclical so customers can pivot and circle back to the conversation as per their preference without starting over. Human to human conversations themselves are not linear and neither should conversational interfaces.
From release and forget, to iterating and tuning
Many organizations that build virtual assistants invest in upfront research and design to understand the customer journey and context. They sometimes, however, drop the ball on iterating and fine-tuning the experience after releasing the virtual assistants to actual customers.
It is crucial for organizations to monitor and evaluate actual conversations to really understand what is working and what isn’t. Reviewing user sessions to investigate errors and determine how to improve the experience should be an integral part of an ongoing sustainment plan. Continuous iteration or ‘bot tuning’ is another critical practice for maintaining a balance of necessary intents and their training data. Tuning could involve various activities like adding, removing, or modifying utterances. Removing intents that don’t add value is just as important as creating new ones.
This results in customer experiences that are as seamless and as simple to navigate as possible. It also increases customer engagement and containment within the conversational experience.
From pre-defined answers, to Natural Language Understanding and Conversation Design
At its core, conversation design aims to mimic human conversations to make digital systems like virtual assistants easy and intuitive to use. The challenge is to make interactions with these systems feel less robotic by understanding the context and purpose of the customer in order to direct them to relevant solutions.
Many organizations, however, still employ hard-coded or rule-based pattern matching with small rule-sets for their conversational interfaces. This results in higher abandonment rates, low engagement, and perceived project failures.
Natural Language Understanding (NLU) technologies utilize machine learning and training data that allows them to understand user utterances without the need to manually hard code all the pattern matching logic. NLU platforms also provide hooks into domain-specific knowledge bases and forums.
Download Guide: Conversation Design and How to Approach It.
By integrating and maximizing the power of NLU platforms, organizations can enable virtual assistants to respond to human queries efficiently and effectively, improving customer engagement and providing an overall positive customer service experience.
From disjointed multiple bots, to a seamlessly integrated omnichannel experience
With an increase in the development of virtual agents, some larger organizations are facing a new challenge. Individual departments are creating conversational interfaces with a narrow scope of handling queries related to very specific use-cases or business functions such as HR or IT. As a result, accessing and discoverability of the numerous virtual assistants becomes a challenge for users.
When appropriate for their situation, organizations can overcome this challenge with the introduction of a “master virtual assistant”. This assistant can be made responsible for handling a range of tasks for the customer by understanding their intent and routing the request to the use-case-specific virtual agent. For example, a financial institution may have separate chatbots to handle commercial and consumer mortgage use cases and a master chatbot that seamlessly manages the interactions across them.
Read more about which processes that could be automated for HR with help of AI.
Is your organization ready to level up its conversational AI experience?
Organizations need to remember that launching a virtual assistant isn’t the destination but a journey. It’s essential to keep in mind that success with conversational AI depends on more than just technology. An elegant conversation design based on research and continuous optimization is also crucial to make virtual assistants more intelligent, intuitive, and engaging.
Whether you’re looking to develop the knowledge and capabilities to scale your conversational AI strategy in-house or find a partner to work with — MOC is available to assist.
Take the first step in leveling up your CAI experience
It’s 2022 — Time to Level up your Conversational AI Experiences was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.
In this post, I am going to talk about the different metrics that we can use to measure classifier performance when we are dealing with unbalanced data.
Before defining any metric let’s talk a little bit about what an unbalanced dataset is, and the problems we might face when dealing with this kind of data. In Machine Learning, when we talk about data balance we are referring to the number of instances among the different classes in our dataset, there are two cases.
When it comes to the distribution of classes in a dataset there could be several scenarios depending on the proportion of instances in each class. Let’s look an example using a binary dataset.
The figure above illustrates the feature distribution of two different classes, As it can be observed instances belonging to the red class is more frequent than the blue class. The class more frequent is called majority class while the class with less samples is called the minority class.
The last plot is an example of an unbalanced dataset, as we can see the class distribution is not the same for all classes. If the data distribution among classes were similar the dataset would be a balanced dataset, the following up image shows this case.
In this case, the scatter chart above shows the data distribution of two classes, in this scenario the amount of instances for each class is similar.
Problems with un balanced data.
We always want to work with perfect data, that is, working with balanced datasets. There are several problems when dealing with unbalanced datasets, when it comes to modeling the most significant might be the bias problem. In this case, since there are more instances belonging to one of the classes the model will tend to bias its predictions towards the majority class.
Another issue that arises when we are dealing with unbalanced data refers to the metrics we are using to measure model performance. And this is the main problem this post is going to tackle.
To good to be true
When it comes to unbalanced datasets we should be really careful about metric choosing and interpretation. For example, let’s say that we have a dataset with 90 instances belonging to one class and 10 to the other one. If we have a classifier that always has the same output, let’s say the majority class we are going to get an accuracy of 90 % which many might think is something great, but the reality shows that the model is unable to distinguish between the two classes, the main task of any classifier.
We can use several metrics to measure performance, such as Precision and recall. In the link above there is a post related to this metrics. One of the limitations of those metrics is that they use one single value for threshold, so the information that they provide is restricted.
Thresholds and classifiers.
When we look at the output of classifiers we usually see a discrete output. however, many classifiers are capable of showing the probabilities instead. For example, in scikit-learn this method is usually called predict_proba it allows us to get the probabilities of all classes.
By default, the threshold is usually 0.5 and is tempting to always choose this value, however, thresholds are problem dependent and we have to think what would be the best option for every case.
Using charts for model comparison.
We already talked about the importance of choosing the right threshold for each problem, to do so it is common to use some charts, in this case I am going to walk through two of them, the precision recall (PR) curve and Receiver Operating Characteristic (ROC) Curve.
The ROC curve shows the variation of true positive and false positive rate for different thresholds. We can use this plot to compare different models. Let’s take a look to the following up example.
The chart above shows the ROC curve for two different models. The x axis show the False positive rate while the Y-axis shows the True positive rate, in this case we can see how the DecisionTreeClassifier seems to be slightly better than the GradienBoostingClassifier since the rate of True Positive is higher than the True Positive rate gotten by GradientBoostingClassifier. However, in this case, both algorithm are quite good and there is not to much information to extract from this plot.
Precision Recall Curve
We can use the PR Curve to get in at glance much more information about the model, the following chart shows the PR curve for the models beforementioned.
The PR curve shows the variation of Recall and Precision through different threshold values. The chart above illustrates the PR curve for two classifiers, in this case, what we are looking at is the precision and recall values, which allows us to make a better comparison. We can observe that the PR curve remains similar for both classifiers for most thresholds and just after the 0.8 value of recall both curves suddenly drop to a minimum of 0.825 in precision.
Now, to choose the right model we have to take into consideration the use case, or the problem case. In a problem where it is important to keep the number of false negatives low (high recall), it might be convenient to choose the Decision Tree Classifier rather than the Gradient Boosting Classifier.
Although the last chart does not allow to make a more realistic comparison, the main idea is to always take into consideration the application of the model. If we want to distinguish between malignant and benign tumors it will be always more convenient to have high recall (low false negatives) since we do not want to diagnose a malignant tumor as benign. In this particular case, this error can put patients’ lives at risk.
Unbalanced data and problem definition
It is important to notice that the last graphic is showing precision and recall values for different threshold values, this is excellent to measure models trained on unbalanced data. Precision and Recall metrics can be seen directly on the graph, thus evaluating the false negative and false positive ratios at a glance.
However, there is one important point to take into consideration, defining the positive class, The metrics that we went through are based on the definition of a Positive class an a Negative class. Let’s go through this classic example once again. Let’s imagine an experiment where we have 100 samples, and let’s say that 90 samples correspond to the class Benign and 10 samples Malignant. There is an imaginary classifier, and in this case let’s say that our positive class are the benign tumors. After training the classifier we get something like this.
TP = 80, TN=0, FP=15, FN=5
Recall = 0.94
Precision = 0.84
As we mentioned before, for this particular case, in which we need to distinguish between malignant and benign tumors, it is important to get a high recall, however, that is true just because we are interested on identifying malignant tumors.
In the last example, we defined as positive class the majority class, thus is tricky to use these metrics to measure performance, since we are having a high number of True Positive only with the problem definition. The numbers have shown that the model is uncapable of detecting malignant tumors (the negative class) since we are getting a true negative ratio equal to zero. The main take away in this case is that clarity in problem definition is paramount.
If you want to keep in contact with me and know more about this type of content, I invite you to follow me on Medium and check my profile on LinkedIn. You also can subscribe to my blog and receive updates every time I create new content.
Model Performance and Problem Definition when dealing with Unbalanced Data. was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.
A look at LLMs and their popularity
Advances in natural language processing (NLP) have been in the news lately, with special attention paid to large language models (LLMs) like OpenAI’s GPT-3. There have been some bold claims in the media — could models like this soon replace search engines or even master language?
But what exactly are these large language models, and why are they suddenly so popular?
What’s a language model?
As humans, we’re pretty good at reading a passage and know where the author might be heading. Of course, we can’t predict exactly what the author will write next — there’s far too many options for that — but we notice abrupt changes or out-of-place words, and can make a stab at filling in endings of sentences. We intuitively know that a message saying “I’ll give you a call, how about” is likely to end with “tomorrow” or “Thursday”, and not “yesterday” or “green”.
This task of predicting what might come next is exactly what a language model (LM) does. From some starting text, the language model predicts words that are likely to follow. Do this repeatedly, and the language model can generate longer fragments of text. For all the recent interest, language models have been around for a long time. They’re built (or trained) by analysing a bunch of text documents to figure out which words, and sequences of words, are more likely to occur than others.
One method of building LMs called n-grams has been around for a long time. These models are quick and easy to build, so people have trained them on different kinds of text. Examples include text generated from Shakespeare: “King Henry. What! I will go seek the traitor Gloucester. Exeunt some of the watch. A great banquet serv’d in;” and from Alice in Wonderland: “Alice was going to begin with,’ the mock turtle said with some surprise that the was”. Train one of these models on something else, like articles from the Financial Times, and the model will predict an entirely different style of text.
N-gram models aren’t good at predicting text that’s coherent beyond a few words. There’s no intent or agency behind what they’re saying; they create sequences of words that might seem sensible at first glance, but not when you read them closely. They’re simply regurgitating patterns in the training data, not saying anything new and interesting. These models have mostly been used in applications like autocorrect, machine translation, and speech recognition, to provide some knowledge about likely sequences of words into a bigger task.
The emergence of large language models
There’s always been a drive to use more and more data for training AI models, and LMs are no exception. In the past decade, this has only accelerated. Training a model on more text means the model has potential to learn more and more about the patterns in language. More data is one part of the ‘large’ in ‘large language models’.
The second part of ‘large’ comes from the size of the models themselves. The past 15 years has seen neural networks as a popular choice of model, and they’ve got larger and larger in terms of the number of parameters in the model.
GPT-3 for example, has 175 billion parameters and is trained on around 500 billion tokens. Tokens are words, or pieces of words. Most of that text data has been scraped from the web, though some comes from books. The combination of lots of data & large models makes LLMs expensive to train, and so only a handful of organisations have been able to do so. However, they’ve been able to better model much longer sequences of words, and the text they generate is more fluent than that generated by earlier LMs. For example, given an initial text prompt to write an article about creativity, GPT-3 generated the following as a continuation:
The word creativity is used and abused so much that it is beginning to lose its meaning. Every time I hear the word creativity I cannot but think of a quote from the movie, “The night they drove old dixie down”. “Can you tell me where I can find a man who is creative?” “You don’t have to find him, he’s right here.” “Oh, thank god. I thought I was going to have to go all over town.”
This is far more readable and fluent than the earlier examples, but it’s worth noting that “The night they drove old dixie down” is a song, and not a movie, and it has no lyrics or lines about a man who is creative. These facts are hallucinated by the model because the sequences of words are probable. As readers, we naturally try and infer the author’s meaning in this passage, but the computer has no agency — it really wasn’t trying to say anything when it generated the passage.
How do language models relate to other NLP technology?
NLP is a broad field — language modeling is just one NLP task and there are many other things you might want to do with text. Some examples include translating text from one language to another, identifying entities like names and locations in your text, or classifying text by topic.
To built models for these other NLP tasks, you can’t just analyse a bunch of documents like for language modeling. Instead, you need to have labelled data — i.e. text that is labelled with the entities or topics that you’re interested in. Or in the case of machine translation, text that means the same thing in two languages. Labelling data is time-consuming and expensive, and a barrier to building good NLP models.
Why all the hype about LLMs?
The bold claims about large language models are inspired by some of their interesting emergent behaviour.
First is that these models can be used as a type of interactive chatbot. By learning appropriate continuations of my text input, they can generate appropriate responses in a conversation. The current generation of chatbots are hand-crafted systems with carefully designed conversations, and they take a lot of effort to create. LLMs offer the possibility of chatbots that are simpler to build and maintain.
The second is that because these models have been trained on a lot of data, they can generate a huge variety of texts, including some that are unexpected. Give GPT-3 the text input, or prompt, to translate text into another language and it’s seen enough multilingual text to have a good go at the translation. That’s without ever being explicitly trained to do translation! The ability to recast NLP tasks into text generation ones and use LLMs to do them is powerful.
A third ability is that LLMs can be fine-tuned to different NLP tasks. An LLM has learned a lot about language during its training, and that knowledge is useful for all NLP tasks. It’s possible to make some small changes to the structure of the LLM so that it classifies topics rather than predicts next words, but still retains most of what it’s learned about patterns in language. Then, it’s easy to fine-tune on a small amount of data that’s been labelled with topic and build a topic classifier. This way of building NLP models by first building an LLM on a large dataset (or, more realistically, using one that a large company has built and released) and then fine-tuning on a specific task, is a relatively new way of building NLP models. This way, it’s possible to build NLP models using far less labelled data than if we built the same model from scratch, and is cheaper and faster. For this reason, LLMs have been dubbed ‘Foundation Models’.
But what are the downsides?
LLMs have some interesting behaviours, and many state-of-the-art NLP models are now based on LLMs. But, as they say, there is no free lunch! There are some downsides to these models that need to be taken into account.
One of the biggest issues is the data that these models are trained on. As with the Shakespeare and Alice in Wonderland examples, LLMs generate text in a similar style to that which they’re trained on. This is obvious in those two examples because of the distinct styles. But even when LLMs are trained on a wide variety of internet text, it’s still the case that the model output is heavily dependent on the training data even if it’s not as immediately obvious in the text they generate.
It’s especially problematic when the training data contains opinions and views which are controversial or offensive. There are many examples of LLMs generating offensive text. It’s not feasible to construct a neutral training set (it raises the question, ‘neutral’ according to whose values?). Most text contains its author’s views to a varying extent, or some perspective (bias) about the time and place it was written. Those biases and values inevitably make their way through to the model output.
As in the creativity example above, LLMs can hallucinate facts and generate text which is just wrong. Because of their ease of use and the superficial fluency of the text they generate, they can be used to quickly create large amounts of text containing errors and misinformation.
The impact of these downsides are exacerbated by there being just a handful of LLMs which are fine-tuned and deployed in many different applications, thus reproducing the same issues again and again.
In summary, large language models are large neural networks trained on lots of data. They have the ability to generate text that’s far more fluent and coherent than previous language models, and they can also be used as a strong foundation for other NLP tasks. Yet, as with all machine learning models, they have several downsides that are still being figured out.
What sets your business apart from the competition in the market? Superior products? A greater variety of products? Faster delivery? While these are all important factors, the one factor that will keep your business in the minds of your customers is how you made them feel before, during, and most importantly, after the purchase has been made.
In other words, your customer experience has to be top-notch, and this is where AI chatbots come in.
Chatbots not only make your business more efficient, but they also help in delivering a customer experience that will leave a positive impression of your brand. Also, customers these days don’t mind speaking to a chatbot, according to Small Biz Genius’s report, which says that as long as the bot is prompt in its response, over 63% of the users will interact with it.
So how do you build a chatbot for your website? Well, that part is easy. You follow a few simple steps on how to build a chatbot that we put together.
But then, there are other questions like:
- Which chatbot builder do you choose?
- How much does implementing a chatbot on your website cost you?
- What are the features that you should consider?
- How important is the UI of the chatbot builder platform?
We answer all these questions in this article and offer you a lot more.
We don’t want to keep you waiting, so let’s dive right in.
Kicking things off in this list is the one chatbot solution that we can describe the best since we have been building and testing it for quite a while. Kommunicate is a customer support automation platform helping you build live chat and chatbots for your website.
Traditional customer service needs a serious overhaul, and Kommunicate envisions a solution to this problem using a human + chatbot hybrid model. Kommunicate’s Kompose chatbot builder uses a simple block builder to build your conversations.
You can build a fully functioning chatbot in a matter of minutes with Kommunicate. What’s more, as you are building your bot, you can see a preview of it on the right-hand side.
You can test your bot as you are building it, which is a great feature to have. Kommunicate also scores heavily on the AI & NLP features, where you can understand the intent of the user by adding keywords. Kommunicate provides integrations with AI systems such as DialogFlow, IBM Watson, and Amazon Lex.
The impressive list of integrations also extends to CMS systems such as WordPress, Squarespace, and Wix, CRM systems such as Zendesk, Pipedrive, and AgileCRM, and E-commerce platforms such as Shopify, Magento, Prestashop, etc.
Kommunicate also integrates well with messaging platforms such as WhatsApp, Facebook, Line, and Telegram. Use automation to manually resolve dormant conversations, automatically hand-off conversations to humans in case the bot is unable to answer a question, and do a lot more with Kommunicate.
What we like about Kommunicate:
- Kompose bot builder: Easy-to-use bot builder that lets you build a chatbot in a matter of minutes. It cannot get simpler than this!!
- An impressive list of integrations.
- Stellar customer support, available to you over the phone, email, and live chat, along with detailed documentation.
- Start: $40/month for 1000 Monthly Tracked Unique Users (MTU). Includes two teammates.
- Grow: $100/month for up to 5000 MTU. Includes five teammates.
- Scale: $400/month for up to 10,000 MTU, includes ten teammates.
- Business: Contact our sales team for more details of our plan.
For a detailed overview of Kommunicate’s pricing, visit our pricing page here.
2. Kuki aka Mitsuku
One of the simplest chatbots when it comes to signing up, Mitsuku, aka Kuki, is an AI chatbot that is primarily here to entertain and not do any serious heavy lifting like the other bots on our list. But does that make Mitsuku a “simple” chatbot? Most certainly not. Mitsuku won the Loebner prize four times in the last decade alone.
What is, or rather was, the Loebner Prize? The Loebner Prize was an annual competition in AI that judged bots on how closely they could mimic a human conversation. We took Mitsuku for a spin, and we have to say, the bot was a delight to talk to.
Mitsuku uses the A.L.I.C.E (Artificial Linguistic Internet Computer Entity) and is built on the A.I.M.L (Artificial Intelligence Markup Language), and has more than 80 billion messages in its huge message log. This enables the chatbot to respond to any question asked.
What we liked:
- Super easy to sign up and use.
- The chatbot responded to a lot of our queries and was able to gauge the emotion of the user accurately most of the time.
- Multi-platform compatibility.
3. Bot Penguin
There are a lot of no-code bot builder platforms on the list here, and Bot Penguin is one such chatbot builder where you can quickly build chatbots, primarily aimed at lead generation. You can train Bot Penguin’s bots based on different target use cases. These use cases can vary from answering FAQs to lead generation, scheduling meetings, and even handling the sales counter.
Powered by Natural Language Processing (NLP), the bots powered by Bot Penguin can carry out conversations with users almost as a human would. Bot Penguin also comes with an agent chat interface where the agents can quickly take over a conversation as and when required.
What we liked:
- Chat window on Facebook Messenger. This makes it a lot easier for the users to monitor their social media traffic.
- CRM Integrations: Bot Penguin seamlessly integrates with a host of popular CRM solutions, including ZOHO, Hubspot, and Salesforce. This means you don’t have to enter the leads manually into your CRM.
- Pricing: To create a single bot, Bot Penguin charges you nothing. Even if you are going with the King plan at $5/ month, which gives you access to unlimited bots, it still works out as a value for money.
Bot Penguin Pricing
- Free Plan: Free of charge. One bot and 2000 messages/ month.
- King Plan: $4.17 per month. Unlimited bots and 3000 to 500,000 messages/month.
- Emperor Plan: Should contact sales for price. Unlimited bots and messages.
How easy is it to build a chatbot? There are several players out there in the market that claim to be truly codeless, and they are. But what sets Chatfuel apart is how easy they actually make it to build your own little chatbot. We were able to get up and running by building a chatbot using Chatfuel in less than 15 minutes, and even people who are unfamiliar with building bots can get up and running in a few clicks.
Chatfuel has a really clean interface, which is one of the key reasons businesses have been supporting it since 2015, when Facebook launched its Messenger API. You can use either the Visual Flow Builder or the Block Builder to build your bots using Chatfuel. Each block in the block builder represents a chatbot element, and you can combine different blocks to form “Sequences.”
What we really liked:
- Chatfuel interface: Minimal, clean, and very well designed, Chatfuel has left no stone unturned when it comes to providing a robust user interface. Every section in the dashboard is assigned to a specific purpose, which leaves you with a very good customer experience.
- Different options to suit different customer needs: Chatfuel comes with an advanced level of personalization, having all the features of a chatbot. Keyword detection, automation choices, and the Chatfuel bot lets you build powerful bots that can go into different conversational pathways.
- Affordability: Chatfuel is one of the most affordable chatbot platforms available in the market compared to all the other bot builders on our list. The pricing details have been mentioned below, and, as you can see, they are not so heavy on your wallet.
- Trial Plan: Up to 50 free conversations for $0
- Entrepreneur Plan: Up to 500 conversations for $15/month
- Startup Plan: Up to 1000 conversations for $25/month.
If you are someone who is just starting out with chatbots, having no previous experience coding or building one, then we highly recommend Tidio.
Tidio calls itself an “all in one, powerful, customer service tool,” and we found that it delivered on many of the promises it made. Setting up Tidio is super easy. You just provide the basic information and select from one of the 21 languages available on the dashboard.
In the next step, Tidio will ask you if you want to offer a discount to a new visitor and greet them with a welcome message, and once that is done, you are good to go. The dashboard is simple enough to navigate through, and Tidio lets you easily set “offline hours” when you are no longer available for business.
You can customize the look and feel of the bot in a few simple clicks from the “Channels” tab in the dashboard and can place the bot anywhere on your screen. Tidio also lets you connect your Facebook Business page to your Tidio account in a matter of a few clicks.
What we really liked about Tidio
- User interface: Tidio offers a simple and intuitive interface and is especially useful if you are a first-time chatbot builder. They start out by asking a few basic questions, like “ Ask them if they’d like to order a phone call,” or “Just greet them warmly.”. The visual bot builder is also a breeze to use.
- Features: Tidio has a huge list of features, including a Live Visitors List, Customer Support Metrics, Live Typing, Canned Responses, Viewed Responses, and Notes, among others. Some of these features are only available with the advanced plans, though.
- Integrations: Tidio has an impressive list of Integrations, including ones with Zendesk, Mailchimp, Instagram, Facebook Messenger, Zapier, Google Analytics, and Hubspot, among others.
Tidio has four plans for its users, including”
- Free: $0 /month, includes two operators.
- Starter: $32.5 /month, includes three operators
- Team: $65.83/month, includes five operators
- Scale: $332.5/month, includes unlimited operators
Botsify is a fully-managed, AI-powered chatbot platform that lets you build powerful chatbots for specific use-cases. Claiming to have more than 4 million users and an impressive 83k bots built to date, this chatbot solution is popular among businesses of all sizes. Botsify provides different types of templates for different industries, such as travel bots or sales bots.
Instead of blocks, chatbots here operate in the form of “stories,” and you can teach your bot to respond to queries that your customers might potentially have. More and more of these stories lead you to build story trees, and you can easily make changes to these trees from the user interface.
Botsify offers bot creation capabilities for a wide array of services, including messaging platforms such as Facebook Messenger and WhatsApp, SMS, WhiteLabel, and also Website chatbot.
What we like about Botsify:
- Ease of bot creation: The UI may not be as polished as some of the other bot platforms that we are featuring on this list, but Botsify makes it super easy to create a bot.
- Collection of templates: Botsify lets you choose from a predefined list of over 24 chatbot templates, and this lets you customize your bot to a very high degree.
- Impressive analytics: Botsify’s chatbot analytics are quite impressive. You can have a real-time look at various metrics such as the bot with the most conversions, bots having the most conversational hours, etc.
- Free plan: 14-day free trial with unlimited chatbot users, analytics and reporting, and a dedicated account manager.
- Do It Yourself: Plan starts at $49/month and gives you access to 2 chatbots and up to 5000 users per month.
- Done for You: This plan starts at $149/month and gives you access to 5 chatbots and up to 15,000 users per month.
Aivo markets itself as a customer service solution that gives you quick solutions, lets you manage your own time, . It also provides personalized answers to all your queries. Aivo has divided its offerings into three umbrellas — AgentBot, Engage and Live.
AgentBot is the conversational AI offering of Aivo that lets you grab the audience’s attention. How? With personalized ads, measure customer satisfaction, and evolve using customers’ experience on the site.
Engage lets you create WhatsApp campaigns. Without any code, you can drive high-quality conversations with customers at scale. This helps you craft effective marketing campaigns.
Live lets you add agents to your conversations with the customer and resolve complex customer queries in real-time. With Live, agents can offer more optimal solutions to customers in less time and also create customized customer support groups.
What we liked about Aivo:
- Detailed analytics: Aivo comes with an agent monitor where you can see the details of each ongoing conversation. There is also a chat report which allows you to analyze the service content and quality.
- Open to feedback: Aivo is always open to receiving constructive feedback and implementing them in its products. This has been re-iterated in multiple reviews online and customer testimonials.
- Starter: $75/month plus an additional $0.18 per conversation. The starter plan includes features such as specialized CX analytics and unlimited live chat agents.
- Advanced: $479/month plus an additional $0.15 per conversation. The advanced plan all the features of the starter plan and additional features such as conditional automation rules and satisfaction measurement.
- Business and Enterprise Plans: To get more information about these plans, you need to contact Aivo’s sales team. You can do so by visiting their pricing page here.
The name Pandorabots might be familiar to you if you have been reading through this entire list, and it should be since these are the people behind Kuki/Mitsuku, which we profiled earlier. Pandorabots is an open-source chatbot framework with which you can build AI-powered chatbots for the web, mobile applications, and messaging apps like Slack and WhatsApp. Pandorabots is based on AIML(Artificial Intelligence Markup Language), which you can use to create sleek, conversational bots.
The “Chat Design” feature lets you visually create the questions and answers for your chatbot. There is, however, a minor catch when it comes to designing your chatbot flows. During this design, it is important to handle context, which may not be possible if you don’t know how to code in AIML. Thus, creating a good chatbot in Pandorabots requires you to have a firm grasp of AIML. The conversations in Pandorabots are always initiated by the user, which is a drawback. Also, chatbots built using Pandorabots cannot handle multiple languages at the same time.
What we liked about Pandorabots:
- Easy-to-use interface, Pandorabots provides both a visual editor for marketers and a code-based editor for hardcore developers.
- Open-source framework, which means you can play around with the code of your chatbot. Really useful for those interested in customization.
- Ability to understand the intent of the user and reply to it accordingly.
Pricing of Pandorabots
- Free plan: Free of charge, no access to API or third-party channels.
- Developer plan: $19/month for 10,000 channel messages/ month and only Email support.
- Pro Plan: $199/month for 100,000 channel messages/month, with Email, chat, and phone support.
- Enterprise plan: Varies according to users.
If you want a chatbot builder that is tailored to meet your customers on Instagram, Facebook Messenger, or SMS, then you might want to check out ManyChat. Manychat boasts of some very impressive features such as message broadcasting, drip messaging, A/B testing, etc. The chatbot platform’s user interface is quite sleek, with a dashboard that is super easy to use. There are two kinds of interfaces for building bots using Manychat — The Basic Builder and The Flow Builder.
The Basic Builder presents all the messages of a certain flow are organized and presented in a predefined order. This makes it easy to keep all your bot messages organized. The Flow Builder is more of a visual drag-and-drop bot builder that lets you connect your messages and actions with each other. This can get tricky if you have a lot of incoming content for your bot, but Manychat allows you to switch to Basic Bot builder any time while building the bot. With ManyChat, you can easily ask questions and save the user’s response to a custom field. Using this, you can send broadcast messages to specific user segments.
What we liked about Manychat
- Audience tab: This gives an overview of all the users who have interacted with your chatbot, which is really useful for your sales team when they make cold calls.
- Well-organized Sequences: The sequences in ManyChat are very well organized, and you can see the entire sequence in one overview window. You can also see which sequence is sent out at which particular time, which is a neat feature to have.
- Built-in live-chat integration.
- Free plan: $0 charged, access to over 1000 contacts, and can collaborate with 1 team member.
- Pro plan: $15/month, access to unlimited contacts, and can collaborate with unlimited team members.
If your customers are primarily on WhatsApp, then WATI is a solution that you should really keep on your shopping list. With WATI, you can set up a WhatsApp chatbot without writing a single line of code in a matter of minutes. WATI also lets you automate nurture sequences. This is an important marketing automation feature to have, considering the fact that WhatsApp messages have over 90% open rates. Another neat feature that WATI offers is Broadcast Messages. This comes with a catch, though, since you need to have an approved template before you send out a Broadcast message.
Broadcast messages on WhatsApp also come with interesting analytics, and you can see the details of your message like Sent, Opened, Read, Replied, Failed, etc., just like an email. WATI has integrations with E-Commerce platforms such as WooCommerce and Shopify, and they are adding more integrations as we make this list. WATI also has tools to assist your sales team, such as Cart Recovery. Now, if a customer has filled out their phone number while purchasing an item before dropping off, you can use WATI to remind them of their purchase. WATI also integrates with CRM such as Zoho, Hubspot, and Zapier.
What we liked about WATI
- Automated review of template messages.
- The ease with which you can send customers notifications over WhatsApp.
- Reports like how many people received messages and how many did not.
- Monthly plan: $49/month, which includes 1000 free conversations and five agents.
- Annual plan: $40/month, which includes 1000 free conversations and five agents.
One of the oldest players in the market, Chatbot.com, is an all-in-one platform that can be used to build chatbots across multiple platforms. The interface of Chatbot.com is a visual drag and drop builder, which we have seen across many players that we have reviewed in this list. Each of the chatbot elements is called Interactions, and a single Story can have an endless number of interactions. Although the Flow builder makes it easy to build chatbots, it can also get a lot complicated if you are building large chatbots.
This is where Chatbot.com excels by providing access to additional Flows. When you click on a new Flow, it opens a new window, and you can start building your bot from here. Chatbot.com also gives you a standard “Test your Bot” feature, with an add-on that is quite interesting. You can start testing the bot from any random spot on the flow by clicking on the “Start testing from here.” option. Chatbot.com also provides decent analytics, where you can see all the users who have interacted with your bot. You can see details such as the customer’s time zone, the channel they used, their email address, and even add custom fields.
What we liked about Chatbot.com
- Flow builder is so easy-to-use that you can be up and running with a bot in a few minutes.
- Ability to store user info into custom fields.
- Training section where you can see the phrases that your chatbot did not understand and then teach them to your bot.
- Starter: $50/month for 1 active chatbot and 1000 valid chats/month.
- Team: $149/month for 5 active chatbots and 5000 valid chats/month.
- Business: $499/month for unlimited active chatbots and up to 25,000 valid chats/month.
- Enterprise: Contact chatbot.com here to know more.
The name Hubspot is synonymous with Inbound marketing, marketing automation, and, of course, awesome content. In fact, it was Hubspot’s co-founder Brian Halligan who coined the term “Inbound Marketing” way back in 2005. But did you know that Hubspot also builds chatbots? In 2017, Hubspot acquired the company Motion AI and has been building bots since then, calling it Conversations. Hubspot also uses a visual chat builder, and there are four kinds of bots that you can build using Hubspot: Instant Reply Bots, Qualify Leads Bot, Book meetings bot, and Support bot.
When you click on one of these bot types, a chatflow is made with corresponding messages. Before your chatflow goes live, Hubspot also gives you an option to test it on Facebook Messenger. Hubspot does not give you the ability to send broadcasts or sequences, which can be seen as a major drawback. Hubspot chatbot does not provide any analytics options, and neither does it have any eCommerce integrations. With all this information, it looks like Hubspot just decided to offer a chatbot solution as an “add-on” and not as a serious tool to enhance your business capabilities.
What we liked about Hubspot chatbot:
- Easy-to-use interface if you are already used to the Hubspot interface.
- Livechat integration, along with other important integrations such as Integromat, Webhooks, and Zapier.
- Ability to make a chatbot in any language.
Hubspot chatbot pricing
- Hubspot has a free trial version and then the starter plan at $45/month, although this price is for the entire Marketing CRM suite and not just the chatbot solution. The other pricing plans are also according to Hubspot plans.
Giosg calls itself a Conversational Marketing Software that is designed to serve agencies and SMEs, helping increase conversion rates by a significant amount.
These bots can also be used to automate your support and sales functions. Giosg provides an impressive list of ready-to-use templates, including ones for FAQ bots, Demo bots, etc. Giosg uses an advanced Machine Learning (ML) Engine to learn and process the chat messages of users. It then stores them in a Knowledge Base and uses them to give meaningful replies to users’ queries.
One of the major use-cases of Giosg is lead generation bots, which are known to be 4x more effective than static lead generation forms. The chatbot platform also integrates with your favorite CRM and marketing automation software, such as Hubspot, Zapier, Zendesk, Salesforce, etc., among others. In addition to these integrations, Giosg also has an open API which means further integrations, and building on top of Giosg is pretty straightforward.
What we liked about Giosg:
A Knowledge base of responses that you can use to train your bot. You can use historical data and talk to the customers naturally using them.
- An impressive list of integrations.
- Real-time metrics that show you how many conversations the AI has automated.
- Starter: €249 /month, which includes Live chat for one user and integrations with Facebook Messenger and Zapier.
- Business: €459 /month, which includes everything in the Starter plan and access to the Full Template Library.
- Enterprise: Details of this plan vary based on the user, and you can find out more here.
The last chatbot platform on our list comes recommended by none other than Neil Patel! MobileMonkey can be used to build chatbots primarily for websites but also for Facebook Messenger, Instagram and SMS. The chatbot flows in MobileMonkey are called “Dialogues,” and the elements that are presented to the users are called “Widgets.” MobileMonkey does not have a visual drag-and-drop Visual Flow Builder, which is a small hiccup. This is because if you have several options for your chat flow, these Widgets will get stacked under each other, and you can’t get a clear overview of the bots you are building.
The way leads are presented in MobileMonkey is quite good, and once you click on a lead, it will display all the requisite details such as “Last Seen” and “Email address.” Moreover, you can store users’ responses in custom fields and then use these custom fields to create user segments. This helps in creating broadcast sequences. With MobileMonkey, you can also create a chatbot in a language of your choice. However, you cannot translate your chatbot to different languages.MobileMonkey provides good analytics with total contacts, daily contacts, and sessions, unanswered questions, and triggered keywords. There are also a total of 23 templates that you can use, all of which are free.
What we liked about MobileMonkey:
- A large number of pre-built ready-to-use templates.
- Omnichannel widget. This marketing tool shows a Facebook Messenger widget when a user is logged in and a native website widget when a user is logged out.
- Ability to analyse the questions to which the chatbot couldn’t respond with an answer. With this, you can improve the chatbot by providing relevant answers.
- Instachamp VIP Edition: Free plan, includes 250 contacts and 10 MobileMonkey branding-free sends.
- Instachamp platinum: $9.95/month, includes 1000 contacts and 500 MobileMonkey branding-free sends.
- MobileMonkey Startup edition: $119/month, includes 3000 contacts and 1000 MobileMonkey branding-free sends.
And there you have it. Fourteen of the top chatbot builders profiled according to their strengths and weaknesses.
We had a tough time deciding which is the best one among these that suits your business. This is because each use case is different, and while some had good integrations, others provided good analytics options.
At the end of the day, we would recommend going with a chatbot provider who gives you an all-rounded solution that gives you maximum value for your money; we would recommend Kommunicate.
Originally Published on https://www.kommunicate.io/ 09/06/2022.
Top 14 AI Chatbot Platforms for Business [2022 edition] was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.
Let’s talk money.
Because you probably read it everywhere — Chatbots are great for lead generation. Chatbots are very cost-effective.
And you’re convinced of their usefulness.
But are you really sold?
Not until you get the answer to the most prominent question you have as a decision-maker — “How much money do I need to spend?”
Well, the answer is — it depends.
The point is, if you want a chatbot solution, you can even get it for free today. But it won’t be right for you if it doesn’t have the features or solutions that your company seeks. Or if the solution isn’t aligned with your vision. Or if it doesn’t address your use case.
So naturally, a lot of factors should be considered when you talk about chatbot costing.
It’s necessary for you to have a holistic outlook of –
- What are the chatbot pricing plans you should know about?
- What are the crucial factors you should consider?
- Are chatbots worth the investment?
Let’s talk about these topics in detail.
What are the types of Chatbot Pricing Models?
The chatbot market has evolved over the years. It has birthed new chatbot pricing models based on the features and other add-ons that a business might need in their chatbot. Fortunately, you’ll find numerous options and you can pick one based on your business needs and sizes. These pricing plans are:
1. Free Chatbot
A lot of vendors offer chatbot software for free. This is perfect if you’re a new business with a limited team and budget, and looking to get acquainted with the technology. A free chatbot will help you evaluate the tool, understand how to leverage its basic features, build the conversation flow, and optimize the bot. Remember that a free plan differs from a free trial which is offered just for a limited time frame.
Only after you reach a point of awareness can you move forward and decide whether you want to purchase the product or continue using it for free. A free plan, for example, that of WotNot’s, includes features such as the bot-builder, analytics dashboard, and limited integrations.
Some top free chatbot providers in the market include:
Since this is a free plan, there’ll be a limit on the number of conversations you can have, the number of agent accounts you can have, custom integrations you can have, and so on. But it works best for a pilot project where you can check how the software works for you.
2. Subscription chatbots
A free plan can only offer so much. Subscription chatbot pricing delivers more features that are useful for you to ramp up automated conversations on all your platforms. The type of subscription pricing includes:
Many chatbot providers offer affordable pricing plans that start at $15 and can go up to $99 per month. This would be ideal if you are a small business with a limited team or fewer than 100 employees. You’ll also have comparatively more features at your disposal. Some examples of affordable pricing include:
As your business grows, you need to keep up with the demands and influx of growing customers through automation. Your chatbot will need features that allow you to provide automated conversations at scale along with advanced integrations.
Mid-market pricing works best when you have a team of around 500 employees and are looking to streamline complex customer transactions and conversations. Some chatbot platforms you can consider are:
3. Enterprise chatbots
Enterprise chatbots include all the features and additional customizations that an enterprise seeks in their tools. This type of pricing usually applies to large enterprises having over 500 employees.
The pricing depends on the extent of customization required by the enterprise. The vendors and enterprises get in touch via a live demo and come to a pricing conclusion based on the custom requirements of the enterprise.
Enterprise plans typically include features of all the pricing plans and a few advanced ones. In addition to that, you can choose to customize your chatbot based on your business requirements.
Some top players in the market and the advanced features they offer in their enterprise plans include:
What are the costs involved in purchasing a chatbot?
Chatbot vendors often include extra costs that range from licensing fees to third-party usage costs. For example, with enterprise pricing, you’re paying a lot more than just features and integrations.
Let’s look at these costs for you to have a comprehensive understanding of the pricing plans:
1. License Costs
Any time an enterprise incorporates a new tool, ensuring privacy is a top concern. And while chatbot brings immense value to businesses, business owners have legitimate concerns about the security, storage of the data collected through chatbots, its protection, and accessibility.
According to a study by Accenture, 45% of businesses are hesitant about implementing chatbots because of uncertain exposure to new privacy, security, legal, and regulatory aspects.
As a result, chatbot vendors often include license costs. These costs are those that provide legally binding guidelines for the use, distribution, and installing the chatbot software on the client’s server. By paying for the license costs, the client mitigates the risks associated with data privacy and prevents hacking and phishing attacks.
2. Installation Costs
With a license, costs come installation costs. This is the one-time fee that you would need to pay the vendor for an on-premise installation. Think of installation costs as the costs associated with installing an app on your phone or desktop.
By paying installation costs, you are waiving off the costs you’ll otherwise incur in hiring in-house developers to get a fully customized chatbot up and running on your platforms.
3. Chatbot Software Platform Costs
This is the basic subscription fee that you pay for using the chatbot platform. This would depend on the platform you choose and the inclusions in each pricing plan.
Most chatbot platforms follow the freemium pricing method, where users have access to free plans and acquire premium features by paying for them.
For instance, WotNot’s pricing structure is divided into:
- Free plan
- Startup plan
- Business plan
- Enterprise plan
4. Conversation Design Costs
Since chatbot is a marketing tool, it’s essential to understand the nuances of chatbot conversations, chatbot marketing techniques, and ways to leverage the script to have valuable outcomes from the same.
If you opt for a business or enterprise plan, you can rely on the chatbot vendors for designing and scripting the chatbot. These costs are allocated towards hiring a graphic designer and a creative content writer who can understand your business objectives and create the creative content for your chatbot flow.
Here is an example of how a chatbot can handle multiple kinds of conversational flows for a financial company that deals with mutual funds.
5. Chatbot Development Costs
Along with someone creative, you need someone with the technical know-how as well — someone who can implement the output of the conversation designer. A chatbot developer is responsible for building and deploying the bot on multiple platforms.
This chatbot development cost can range from $500 to $2500.
Chatbot developers have a good understanding of what works best in every platform and clone the bots based on the platform specifications. The services of a chatbot developer are essential to optimize your bot, handle minor tweaks, and ensure seamless deployments.
6. Custom Integration Costs
Your chatbot platform may already provide some custom integrations but say, a company is using a custom CRM software or a live chat system that requires custom integration. In that case, these integrations are charged on a time and material model. With these additional integrations, the overall solution will be more advanced and personalized to your needs.
7. Ongoing Support and Maintenance Costs
You pick any tool — the work doesn’t end once it’s up and running.
The tool constantly evolves to accommodate additional requests or updates for better functioning.
For example, you check the chatbot analysis and conclude that the conversations are abandoned at a particular question. As a result, the chatbot agency would have to change the flow to ensure the completion of the conversation. These are called the support and maintenance costs.
You can only predict these costs once you see the bot in action for a few months. You’ll realize when you need an AI chatbot, or an FAQ builder, and have to request these integrations from the chatbot provider. The cost could either be a part of the subscription fee as a feature where the customer gets 10 hours of support each month or it could be an add-on.
8. Usage Costs
Usage costs include any additional charges that you pay to a third party. This can include the costs of a WhatsApp business API, SMS costs, NLP engine costs, etc.
For example, if you want to deploy a bot on WhatsApp, you’ll need to work with an official WhatsApp business solution provider that will help you access the API. They offer around 10000 messages free per month. However, they’ll charge you extra if you exceed the plan limit. You will have to incur the set-up costs and cost-per-conversation (which usually varies based on different regions and providers).
Factors that Determine the Chatbot Costs
Now that you have an overview of how chatbots are priced, you need to figure out how you should decide on the costs to pay. To do so, ask the following questions before deciding:
What is your Business Goal?
Your business goal won’t just decide your chatbot pricing but also your use case, the type of bot you require, and the channels you want to cover. So ultimately, it all boils down to one simple question, “why the need for a chatbot?”. Most businesses aim to achieve the following long-term outcomes from a chatbot:
- Increased automation in key tasks
- Higher site conversions
- Increased sales
- Higher customer engagement
- Reduced operational costs
You can have more than one end goal but this clarity will help you draft a pilot project for your chatbot and also decide how much investment you should allot and the ROI you should expect.
Identifying these goals will decide the features you’ll need, platforms you should consider for deployment, and integrations that will be beneficial.
What is your use case?
Not every chatbot you see can achieve the same outcome. They are designed for specific use cases that can range from demand generation to customer support. As mentioned above, your use case will be determined by your business goal.
Your use cases will define the flow and integrations that will eventually decide the costs associated with your chatbot. Some of the most common use cases of a chatbot that you might want to consider are:
- Demand Generation Chatbot
- Appointment Scheduling Chatbot
- Product Recommendation Chatbot
- Order Placing Chatbot
What type of Bot do You Require?
While the function of every chatbot is driven by the need to drive conversations on websites, the way they achieve their goals is different. There are different kinds of chatbots having varied costs.
Think of a rule-based chatbot as a decision tree formulated in the form of conversation. Rule-based chatbots have a predetermined flow. The user is made to choose an option to take the conversation further so that it doesn’t deviate from the flow or the rules assigned to each question.
An AI chatbot possesses machine learning capabilities and is programmed via an NLP (Natural Language Processing) engine. AI chatbot converses with users by understanding the intent of the question and provides answers based on the intent. With time, AI chatbots get better at recognizing the intent and learn to give out accurate answers to customer queries.
It’s unnecessary for your chatbot to be rule-based or an AI chatbot. It can be a combination of both. Rule-based and AI chatbots are at different ends of the spectrum. Hybrid chatbots are a way to combine them to leverage the benefits of both.
Such chatbots have some rule-based tasks and an understanding of intent and context. It allows you to add AI elements only where you require them.
You must wonder, “that’s great, but how does this affect my pricing?”
Here’s the deal: Rule-based chatbots are straightforward and don’t require any NLP engines or the expertise to build and deploy chatbots on your website. But like we mentioned the usage costs above, what makes chatbots ‘AI’ is their NLP engine integration like Dialogflow or IBM Watson.
These NLP engines often have extra costs associated with them that you’ll have to consider, and the costs of talent hired to build them. For example, IBM Watson charges $0.0025 per API call in its standard plan.
So naturally, if you decide to go for an AI chatbot, then you’ll have to pay 2–3 times higher than the rule-based chatbot.
How complex is the flow to build this chatbot?
Although we discussed the reasoning behind why AI chatbots are expensive, we cannot assume that all rule-based chatbots will be affordable. It takes considerable effort to build a rule-based chatbot as well. Especially if it involves a long conversation flow that ensures that the conversation is engaging throughout and leads to a desirable outcome.
It will take a creative writer several hours and expertise to build a highly engaging chatbot along with drafting a compelling chatbot copy that sounds natural and humane.
This leaves us with a rather simple conclusion: the longer and more complex the flow, the costlier the solution.
What is Your Budget?
Even if your business needs an AI chatbot or a complex chatbot flow, you cannot have it if you don’t have the budget for it. Most times, the management may be reluctant to invest in new technology and have a legitimate fear of not getting the expected ROI.
But the best part about the plans that we discussed so far is that there’s a lot of flexibility in pricing options. There’s a chatbot plan for every budget from $0 to $1000.
If you have a minimal to nil budget, you can always opt for a free plan or a basic plan that gives you access to just the critical features. It is also a great way to get started with a chatbot tool. You can use it and measure its progress so that you can ultimately pitch to ask for a higher bid from the management.
After using the tool and scaling up your business, if your budget increases marginally, you can opt for a business plan that is inclusive of almost all the features you would need in a chatbot.
What Channels Do You Want to Cover?
A lot of businesses want to establish a presence on popular platforms and connect with customers on messaging apps they use. If you’re deploying a bot on a website or Facebook, you don’t have to incur any additional costs.
However, going back to the usage costs, deploying a bot on WhatsApp or SMS will cost you extra, which is payable to third-party vendors.
If we take the example of WhatsApp, you first need to avail an API from a WhatsApp business solution provider like Twilio or 360dialog.
For WhatsApp, in the USA, Twilio doesn’t charge for the first 1000 conversations each month and then charges a $0.005 flat fee per interaction (period of 25 hours). Similarly, businesses can send an SMS to a local number for $0.0079. The pricing changes for each country and API providers.
What is the Cost of the Workforce?
By now, you must’ve realized the importance of location when deciding your chatbot cost. The country where you get your chatbot developed will largely affect the chatbot pricing, primarily because of the differences in hourly wages.
In the US, an employee will cost you around $50/hour. Now consider all the talent that goes into building a chatbot — the developers, the marketers, the writers, and the designers. A chatbot solution in the US can be expensive, whereas you can pay three times less and get the same solution from a country in Eastern Europe or India where an employee will cost you less than $25/hour.
What is the Volume of Conversations that the Chatbot will Automate?
Any brand that wants to implement chatbots can expect an increase in conversation volume capability. Since chatbots work on the SaaS model, the more conversations they handle, the more server power the chatbot platform needs to have. This volume largely depends on your website and social media traffic.
The pricing plans will typically have an upper limit on the number of chats. Based on your current support volume, you need to figure out how much will be diverted towards chatbots and how many requests are they expected to handle. It will help you figure out your chat volume. If your chatbot capacity needs to be scaled to accommodate a higher volume, the cost will shoot up accordingly.
What Integrations do you Require?
Integrations are a bare necessity for chatbots. If you build a scheduling chatbot, you’ll need a scheduling tool integration like Calendly, or if you build a lead generation chatbot, you need a CRM integration like Salesforce.
It is the interconnectivity between these tools that makes your chatbot solution comprehensive. Approximately, the cost of an integration can vary from $1000 to $10,000. These prices depend on the complexity and level of customization needed to integrate a tool into the chatbot which will add to your costs.
Chatbot Cost-Benefit Analysis
Finally, the moment you’ve been waiting for. The moment of understanding if chatbots are even worth the hassle.
Research suggests that chatbots answer over 80% of the standard queries and save up to $2–5 per interaction. For example, if your chatbot handles 10k chats per month, you save around $50k per month. If it handles more than 20k chats per month, you approximately save $100k per month, and so on.
You already know how chatbots could potentially be beneficial. But you also need to understand the extent to which they’re beneficial in your scenario, albeit with your budget and business goals.
A cost-benefit analysis will help you decide if you really need a chatbot and how much money you should invest in the tool.
So to understand if they can add value to your business, identify the following metrics:
Estimate the Queries that can be Automated
Your agents are at the front line, speaking to your potential customers regularly. Get to know more about the repetitive queries they come across. For example, a mid-size real estate agency can come across numerous questions about their new projects, like:
Once you understand the most common questions that prospects have, categorize them into the level of support they need. Some inquiries require first-level support and some require speaking with a live agent.
In our example, a real estate chatbot can automate most answers like showcasing the latest projects and their amenities with carousels, adding location details with maps, etc. Apart from pricing, the bot takes care of all the straightforward answers. This means that 4 out of the 5 most common questions (80%) of the queries can be answered via a chatbot.
Estimate the Current Time Spent on Answering Inquiries
Continuing with our example, let’s say an agent spent an average of 15 minutes of call duration explaining every basic and minute detail of the project. As chatbots can take over 80% of the load, the average call duration can decrease to 3–4 minutes.
The decreased time would help your agents be more productive and be available for more important calls where they can nurture the leads, talk about pricing, schedule appointments for property viewings, etc. It will also eliminate their mundaneness and allow them to address only queries that actually require their attention.
Calculate the Annual Cost of Handling Inquiries
Your real estate agents are completing a 15-minute call in 4 minutes.
Now consider that they have eight-hour shifts, out of which they spent five hours on calls. With chatbots, your agents are saving up to four hours every day. Not only does this reduce dependency on your workforce but also allows you to be available 24*7, which wasn’t possible before. It also affects the wage you pay annually.
For example, if your agent works for $25/hour for 20 hours a week, you’re paying them $19, 200 annually.
On the other hand, WotNot offers a business plan at $499 per month, which comes down to around $6000 annually.
As for your agents, they can concentrate on other valuable demand generation activities instead which can increase your sales.
According to research, chatbot automation can lead to $23 billion in savings from annual salaries.
Calculate the Business Value Generated
Chatbots don’t just save costs, they generate business value as well. Since chatbots increase conversion rates, they increase the number of hot leads that can be directly passed on to agents.
According to a study by Forbes, chatbots increase sales by 67%. You can calculate your lead generation by determining the lead-to-chat ratio for both manual and estimated automated leads generated via chatbots. You can then compare the two to see the accurate business value generated.
The lead value for a B2B company is usually $2000-$10,000, depending on the industry. So if your chatbot generates 500 leads per month, it means that $100K to $500K in the pipeline is directly attributed to the bot.
A chatbot can cost you anything from $0 to $5000 per month. The pricing is usually divided into a free plan, start-up plan, business plan, and enterprise plan.
Before deciding on the pricing, ensure that you consider all the factors mentioned below:
Calculate a cost-benefit analysis of the tool to get a better understanding of how the chatbot will bear fruits for your business.
Evaluate the results and choose a plan that gives you the highest ROI.
It suffices to say that there’s no one answer to the question, “how much does a chatbot cost?” Numerous players offer diverse plans based on business sizes and requirements. It’s easy to fall into the notion that automation always means good. It won’t help you if you go out of your budget to install a solution that your business doesn’t even need. So it’s your job to assess your needs and figure out how you can best leverage the tool.
While investing in a new tool can always seem intimidating and a gamble, you can always find a way out by signing up for a free plan. You can sign up on WotNot and determine your feature and integration requirements as you use it.
Are chatbots cost-effective?
Yes. Chatbots answer over 80% of the standard queries and save up to $0.07 per interaction. Chatbots automate key tasks and reduces the dependency on agents for mundane tasks such as lead generation and first-level support queries. It saves time and costs of hiring employees for a 24*7 service and helps you scale your customer conversations without hiring an extra workforce.
How is chatbot value calculated?
You can simply compare the number of queries handled by a bot with the current costs spent on answering simple queries. You also estimate the value your bot generates by estimating the extra lead value you can gain from chatbots with the bot installation costs or your subscription costs.
How much time does it take to build a chatbot?
It depends on the complexity of the flow, and the integrations needed. But considering everything, you can build and deploy a bot anytime between 2–8 weeks.
Are chatbots free?
Yes, you can get a free chatbot on WotNot. You can build unlimited bots and get access to the no-code bot builder, live chat, analytics dashboard, and chatbot templates in the free plan.
How much does a WhatsApp chatbot cost?
While downloading WhatsApp is free, it is the WhatsApp Business API for which you will have to pay extra to a WhatsApp Business solution provider. WhatsApp Business API price is now conversation-based. This means WhatsApp now charges businesses per conversation rather than per notification. You can expect a cost of $0.005 flat fee per message sent to the USA.
Originally published at https://wotnot.io.
Everything you Need to Know About Chatbot Pricing in 2022 was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.
The Alexa team demoed the new feature during the event by presenting a scenario in which Alexa uses the voice of a dead grandmother to read a bedtime story to a little boy
Dale John Wong — Alexa will soon be able to talk using a loved one’s voice (even if they’re dead)
That quote from a recent article builds on the digital resurrection post from June 14th. When they say anyone’s voice — they mean it. Take a look at the video below starting at the 1:01:58 mark, which is a different application of the same scenario — resurrecting a lost loved one. If you have not already responded to the poll below, please provide your thoughts.
Originally published at http://frankdiana.net on June 27, 2022.