Starting a chatbot project can seem overwhelming, but following best practices makes the road ahead smoother. Benefit from our team’s experience deploying chatbots with five need-to-know techniques to improve the success of your chatbot implementation.

1. Start with a narrow domain

Don’t try to build Siri. Unless you have a massive budget, and a team of hundreds of engineers, you’re not going to be able to keep up with the likes of Microsoft Cortana, Google Assistant or Siri. Even those teams don’t always get it right.

Keeping the domain of your chatbot narrow to start and setting your users’ expectations about what the bot can deliver is a realistic way to achieve success. Ideas for a first phase could be a bot that delivers company news, gives information about a conference, provides contact details, or answers questions on a specific subject matter. Later, the chatbot can be expanded to have a wider domain.

A good example of this is a Tangowork chatbot implemented at international healthcare company Bupa (read the Bupa case study). When Bupa planned to move their headquarters, they created a chatbot to answer questions about the move, like “when are we moving?” or “do I need to pack my own things?”. After the office relocation, they gradually expanded the bot to answer common day-to-day questions, like “where can I print?”, “where do I get a new Skype headset” or “what’s the number for HR?”.

2. Have one killer feature

What answer does your chatbot have that your employees can’t live without? When your chatbot has a feature that keeps employees coming back again and again, they’re going to turn to it as a resource for other information as well.

At Bupa, now that Cyan (the Bupa chatbot) answers questions about day-to-day work, the number one question it gets asked is “What’s the guest wifi password?” Since the password changes frequently, many employees find that the easiest way to get the password for a guest is to ask Cyan.

3. Design failure carefully

You need to intentionally design what failure looks like, because a chatbot is not always going to be able to give a user the answer they are looking for. That happens for several reasons:

• The chatbot might need more examples to delineate related questions
• The chatbot might need more training on unexpected terms the employee is using
• The chatbot doesn’t contain any answers related to the employee’s question

Every response the chatbot gives will fall into one of the four categories on a tool known as the confusion matrix:

• True positive: the chatbot knows the right answer and delivers it
• False positive: the chatbot knows the right answer, but delivers an incorrect answer
• True negative: the chatbot doesn’t know the answer, and says it doesn’t know
• False negative: the chatbot knows the right answer, but says it doesn’t know

The Tangowork Chatbot Accelerator reduces false positive answers by using a confidence threshold: it only returns an answer if the chatbot is at least 40% sure that it has the correct answer. For answers where the Tangowork Chatbot Accelerator is 40 to 60% confident, it delivers the answer but then asks the user to confirm whether their question was correctly understood.

It’s good to consider each of these scenarios for your bot, and analyze what the chatbot will do in each case. Because failures (incorrect or unknown answers) are going to occur, designing for failure will result in the best possible outcome when it happens. Make sure that the chatbot is giving the best answer possible in each scenario, and steer the user back to supported tasks when needed.

If a user is asking the chatbot for information that it is not designed to provide, redirecting a user back to supported tasks helps the user know what the chatbot can do for them.

Instead of:

Sorry, I don’t understand. Ask me something else.

Try:

Sorry, I don’t understand. I know things about 
Acme Human Resources policies and benefits. 
Try "benefits", "payroll" or "time off".

4. Grow your pilot gradually

At the beginning, a small team of stakeholders and subject-matter experts brainstorms questions and answers that the chatbot will be fielding. Starting the pilot with 10 or 20 people allows the team to review the conversation transcripts and see questions that weren’t anticipated or that the bot is misunderstanding. They can then teach the bot to handle those questions, expand the pilot by another 10 or 20 people, and repeat the process.

As the pilot grows, the percentage of successful responses climbs higher and higher. If you launch at the very beginning, the number of unsuccessful responses will result in user frustration and failed adoption. Once the success rate is in the 90-95% range, the chatbot is ready to launch to the entire organization.

Growing your pilot gradually allows for transcript review, chatbot training and greater insight into user needs, for a high conversation success rate on launch.

5. Review transcripts constantly

Reviewing chatbot conversation transcripts is especially important during the pilot period for your chatbot, but it continues to be an important part of general maintenance. Transcript review allows you to see when the chatbot doesn’t understand a message, or doesn’t have the answer a user is looking for. Fine-tuning the bot by adding more information on a topic, or training it to understand a user’s intention in a particular message allows for continuous improvement.

Conclusion

Apply these best practices to find success as you enhance your employees’ digital workplace with an informed, responsive chatbot.