5 best practices for deploying an employee or HR chatbot

Starting a chatbot project can seem overwhelming, but following best practices makes the road ahead smoother. Benefit from our team’s experience deploying chatbots with five need-to-know techniques to improve the success of your chatbot implementation.

1. Start with a narrow domain

Don’t try to build Siri. Unless you have a massive budget, and a team of hundreds of engineers, you’re not going to be able to keep up with the likes of Microsoft Cortana, Google Assistant or Siri. Even those teams don’t always get it right.

Even teams with hundreds of engineers don’t get everything right. (Don’t worry, they’ve fixed it.)

Keeping the domain of your chatbot narrow to start and setting your users’ expectations about what the bot can deliver is a realistic way to achieve success. Ideas for a first phase could be a bot that delivers company news, gives information about a conference, provides contact details, or answers questions on a specific subject matter. Later, the chatbot can be expanded to have a wider domain.

Keep the domain of your chatbot narrow at the start

A good example of this is a Tangowork chatbot implemented at international healthcare company Bupa (read the Bupa case study). When Bupa planned to move their headquarters, they created a chatbot to answer questions about the move, like “when are we moving?” or “do I need to pack my own things?”. After the office relocation, they gradually expanded the bot to answer common day-to-day questions, like “where can I print?”, “where do I get a new Skype headset” or “what’s the number for HR?”.

“Before our move to Angel Court, the chatbot focused mainly on the office move: things like ‘when are we moving?’ and ‘do I need to pack my own things?’ After the move, we expanded to questions about day-to-day work in the new environment.”
Del Green

Senior Digital Communications Manager, Bupa

2. Have one killer feature

What answer does your chatbot have that your employees can’t live without? When your chatbot has a feature that keeps employees coming back again and again, they’re going to turn to it as a resource for other information as well.

At Bupa, now that Cyan (the Bupa chatbot) answers questions about day-to-day work, the number one question it gets asked is “What’s the guest wifi password?” Since the password changes frequently, many employees find that the easiest way to get the password for a guest is to ask Cyan.

3. Design failure carefully

You need to intentionally design what failure looks like, because a chatbot is not always going to be able to give a user the answer they are looking for. That happens for several reasons:

  • The chatbot might need more examples to delineate related questions
  • The chatbot might need more training on unexpected terms the employee is using
  • The chatbot doesn’t contain any answers related to the employee’s question

Every response the chatbot gives will fall into one of the four categories on a tool known as the confusion matrix:

  • True positive: the chatbot knows the right answer and delivers it
  • False positive: the chatbot knows the right answer, but delivers an incorrect answer
  • True negative: the chatbot doesn’t know the answer, and says it doesn’t know
  • False negative: the chatbot knows the right answer, but says it doesn’t know

Each possible type of response that the bot might give needs to be considered

The Tangowork Chatbot Accelerator reduces false positive answers by using a confidence threshold: it only returns an answer if the chatbot is at least 40% sure that it has the correct answer. For answers where the Tangowork Chatbot Accelerator is 40 to 60% confident, it delivers the answer but then asks the user to confirm whether their question was correctly understood.

It’s good to consider each of these scenarios for your bot, and analyze what the chatbot will do in each case. Because failures (incorrect or unknown answers) are going to occur, designing for failure will result in the best possible outcome when it happens. Make sure that the chatbot is giving the best answer possible in each scenario, and steer the user back to supported tasks when needed.

If a user is asking the chatbot for information that it is not designed to provide, redirecting a user back to supported tasks helps the user know what the chatbot can do for them.

Instead of:

Sorry, I don’t understand. Ask me something else.


Sorry, I don’t understand. I know things about 
Acme Human Resources policies and benefits. 
Try "benefits", "payroll" or "time off".

4. Grow your pilot gradually

At the beginning, a small team of stakeholders and subject-matter experts brainstorms questions and answers that the chatbot will be fielding. Starting the pilot with 10 or 20 people allows the team to review the conversation transcripts and see questions that weren’t anticipated or that the bot is misunderstanding. They can then teach the bot to handle those questions, expand the pilot by another 10 or 20 people, and repeat the process.

As the pilot grows, the percentage of successful responses climbs higher and higher. If you launch at the very beginning, the number of unsuccessful responses will result in user frustration and failed adoption. Once the success rate is in the 90-95% range, the chatbot is ready to launch to the entire organization.

As the chatbot pilot gradually grows, incorrect responses decrease

Growing your pilot gradually allows for transcript review, chatbot training and greater insight into user needs, for a high conversation success rate on launch.

5. Review transcripts constantly

Reviewing chatbot conversation transcripts is especially important during the pilot period for your chatbot, but it continues to be an important part of general maintenance. Transcript review allows you to see when the chatbot doesn’t understand a message, or doesn’t have the answer a user is looking for. Fine-tuning the bot by adding more information on a topic, or training it to understand a user’s intention in a particular message allows for continuous improvement.

Transcript review pinpoints answers the chatbot is missing, which are added for an increased success rate


Apply these best practices to find success as you enhance your employees’ digital workplace with an informed, responsive chatbot.

Summer Chatbot Webinar Series

In July, August and September, sign up for 3 free webinars that explore 3 types of chatbots: Intranet Chatbots, Event Chatbots, and HR Chatbots.

Tangowork Chatbot Consulting

Tangowork’s consultants use research, design and technology to help your organization create better digital experiences. We specialize in intranets, AI assistants and high-performance websites.

The post 5 best practices for deploying an employee or HR chatbot appeared first on Tangowork: Consultants for Intranets, AI Assistants, Fast Websites.


Tangowork Chatbot Accelerator Release Notes

Tangowork Chatbot Accelerator v22: Excel import/export

July 16, 2018

What’s new: chatbot

  • Buttons with links. Now you can add hyperlink buttons to messages. When the user clicks, they’re redirected to a URL.
  • Messages with non-text. Tangowork Chatbot Accelerator now recognizes unsupported messages and responds appropriately. For example, if a user sends a photo, Tangowork Chatbot Accelerator responds with “Thanks for the photo. Unfortunately, I only understand text.” Works for audio, video, images, location, and the Messenger “thumbs up” button. Responses are customizable.

What’s new: management console

  • Excel import/export. Add and maintain content via Excel. You can export existing content to a spreadsheet, make changes, then import it again.
  • People import/export. For private bots, add and maintain a list of users via Excel or JSON. It’s ideal for scenarios where automated synchronization via API isn’t possible.
  • Phone number internationalization. Phone numbers are automatically converted to the E.164 standard when they’re added to the system. For example, (604) 555-1212 is converted to +16045551212. This ensures that duplicates are recognized and ensures that SMS messages are delivered. A “Country” preference in “Bot settings” allows the Tangowork Chatbot Accelerator to infer country code (e.g. “+1”) when none is provided.
  • Default introduction for broadcasts. Set a default introduction for broadcasts, such as “**ACME EMPLOYEE ALERT**”.
  • We moved the “Chat” button to the top-right corner so it’s easier to find.
  • We changed the behavior when you click on intent names. Clicking now shows and hides the intent’s messages. To edit the intent, click the pencil icon.
  • We changed the behavior when you click on intent metadata like sample questions, sort order and list type. Clicking now opens the entire “Edit intent” dialog and highlights the metadata you clicked on.
  • We’ve changed dialogs so that they can only be closed by clicking a button or by clicking a close icon. Previously, clicking outside the dialog closed it, but that led to inadvertent closures.

Bug fixes

  • The prompt to “Train language model” was occurring too often. Now it’s only appearing when it really needs to.
  • Buttons in messages couldn’t be edited in the management console. Now they can.
  • Long intent names were getting truncated on narrow screens. Now they’re not.

Tangowork Chatbot Accelerator v21: Easy entity editing

June 21, 2018

What’s new: management console

  • We completely overhauled working with entities (e.g. in “What is John Doe’s phone number?”, the entity is “John Doe”). Now you can add new entities and tag them within sample questions, all within the management console.
  • Sometimes you need to run custom integrations on a schedule, like for running a nightly import. Now you can.
  • If users ask about a date or time, the “DateTime” prebuilt entity needs to be enabled. Now you can do that yourself in the management console, without asking Tangowork Support.
  • We added a tiny little X to the right-hand side of the search box so you can quickly delete your search term. No more backspace backspace backspace.
  • Now you can instantly change confidence thresholds from within the “Bot settings” page. The default minimum confidence is 40%, but some bots can benefit from dropping it lower. The default confidence for sending user verifications, i.e. “I’m only 45% confident in my answer… did I understand you correctly?” is 60%.
  • We changed the default response setting for new content to “Random” and “No list”. It used to be “Newest” and “List: Cards”, but we found this wasn’t appropriate in the majority of cases.

Bug fixes

  • When you delete content, we used to move sample questions to the “None” category, but that was degrading the accuracy of the AI. Now we just make deleted content vanish, and the AI is much happier.
  • It wasn’t possible to use your iPad to add new content in the management console. We made a few tweaks, and now the iPad compatibility, while not perfect, is much better.
  • We lost the ability to log on to the management console in Safari, but we found it again.

Tangowork Chatbot Accelerator v20: Custom buttons in messages

May 9, 2018

What’s new: chatbot

  • Messages with buttons. Add any button to any message. Buttons can trigger a certain message, a certain intent, or can send quick-reply text.

What’s new: management console

  • Add custom buttons or triggers to any message. A new “Buttons” option on Add/Edit Message lets you define custom buttons or triggers for any message. In Skype for Business or SMS, buttons don’t display, but the button can down-render to a prompt, such as “Type ‘more’ for more info.”
  • Bot settings screen. The new bot settings screen lets you quickly adjust the appearance of your bot across channels. For example, you can change the color, font and icon for web chat.
  • Manual message ordering. For lists of messages, you can now select manual ordering. Drag and drop messages to sort them how you want.
  • To make room for expanded functionality, the Add/Edit Message screen now uses collapsible cards.

Bug fixes

  • Whenever we displayed Cards in Slack, we were showing action buttons twice — before and after the card. It looked confusing, so we eliminated the first set of buttons.

 Tangowork Chatbot Accelerator v19: Custom integrations

April 2, 2018

What’s new: management console

  • Help. Inline help is now available on every page.
  • Content categories. Create your own categories to organize content.
  • Custom integrations. It is now possible to execute any custom code in response to an intent. For example, a user could type “search for expense form” and the Tangowork Chatbot Accelerator can execute that search on SharePoint.
  • Previous button. Users can now page through lists of messages using “Next” and “Previous” buttons. This only affects intents where lists are enabled. (Buttons are not supported on Skype for Business or SMS.)
  • Content export. Export content. This is designed primarily for exchanging content with other Tangowork Chatbot Accelerator installations.
  • Content import. Import content. This is designed primarily for exchanging content with other Tangowork Chatbot Accelerator installations.
  • Parent & child intents. There is now full support for parent & child (nested) intents. This allows for lists of lists.

Tangowork Chatbot Accelerator v18: Confidence thresholds

February 13, 2018

What’s new: chatbot

  • Fewer false positives. the Tangowork Chatbot Accelerator now employs a confidence threshold that a potential answer must meet before being sent to the user. If the chatbot is less than 40% confident (configurable) that the potential answer is the right one, it sends a ‘not understood’ reply instead.
  • User confirmation for medium-confidence answers. If the Tangowork Chatbot Accelerator is 40 to 60% confident (configurable) that an answer is correct, it sends the answer, but then verifies with the user whether their question was correctly understood. The user is presented with a “Yes” or “No” button on platforms that support it, or a prompt to enter “yes” or “no” on text-only platforms.

What’s new: management console

  • Verified answers screen. A new “Verified answers” screen lists questions and corresponding answers that end users have confirmed are correct or incorrect. The administrator can agree, disagree or discard the confirmation.
  • Unrecognized questions screen. A new “Unrecognized questions” screen lists user questions that weren’t understood (i.e. that were below the confidence threshold). The administrator can align the question with the correct answer or discard it.
  • Transcripts now list the confidence under each user question. If a user confirms a medium-confidence answer, it’s noted on the transcript. If an administrator updates the “Verified answers” or “Unrecognized questions” screens, the change is marked on the corresponding transcript.
  • The list of transcripts now scrolls indefinitely. Once you scroll near the bottom of the list, the next 100 conversations load.

Tangowork Chatbot Accelerator v17: Minor changes

February 1, 2018

What’s new: chatbot

  • Long questions rejected. Very long questions aren’t usually understood by the language engine, and they’re usually entered in error anyway. If a user asks a very long question (longer than 140 characters), the bot now responds with, “I have trouble understanding long sentences. Please try again using fewer words.” This response is configurable.

What’s new: management console

  • White space is now automatically trimmed from the start and end of all input on the management console.

Bug fixes

  • Transcripts were sorting by date created instead of date updated, causing some current conversations to be hidden low on the list. Now transcripts are sorted by date last updated.
  • Transcripts weren’t being recorded for Microsoft Teams or Cortana. Now they are.
  • Some errors returned by Microsoft Cognitive Services weren’t being surfaced in the management console, and it appeared the console was hanging. Now those errors are shown.

Release history

  • Version 22: Excel import/export
  • Version 21: Easy entity editing
  • Version 20: Custom buttons in messages
  • Version 19: Custom integrations
  • Version 18: Confidence thresholds
  • Version 17: Minor changes
  • Version 16: Annotate transcripts with notes
  • Version 15: Direct language model training
  • Version 14: Chatbot analytics
  • Version 13: Features for when the chatbot doesn’t know
  • Version 12: Chatbots for Sharepoint, easier content management
  • Version 11: Messages with multiple intents
  • Version 10: Skype for Business improvements
  • Version 9: Transcripts, Subscribe, Test Broadcasts
  • Version 8: Better date support, broadcasts for public bots
  • Version 7: New search technology
  • Version 6: Intent management & embedded web chat
  • Version 5: Enhanced management console
  • Version 4: Skype for Business, image management
  • Version 3: Carousels & buttons
  • Version 2: More preferences
  • Version 1: Content management, broadcasts and language processing

Tangowork Chatbot Consulting

Tangowork’s consultants use research, design and technology to help your organization create better digital experiences. We specialize in intranets, AI assistants and high-performance websites.

The post Tangowork Chatbot Accelerator Release Notes appeared first on Tangowork: Consultants for Intranets, AI Assistants, Fast Websites.