How to Automate Testing of Your WhatsApp Chatbot

Botium delivers again — in this case, automated end-2-end testing of WhatsApp chatbots on real or virtual devices. For the first time it is now possible to have a full enterprise-level test strategy for WhatsApp chatbots.

Automated vs Manual Testing

When it comes to testing WhatsApp chatbots up to now there have been mainly two approaches:

  • Testing manually on a smartphone
  • Testing backend functionality with API Testing

Both approaches are valid and no enterprise-level test strategy should miss any of them. But there are two obvious flaws:

  1. Manual testing is not scalable to the extent that is required to guarantee constant high quality for production-level chatbots
  2. Backend testing does not consider the end user experience

Botium fills this gap with a new connector for testing WhatsApp chatbots on real and virtual smartphone devices.

Building Test Infrastructure

Here is what you need:

  • Some smartphones with SIM cards and USB cables
  • Or Docker + Docker-Compose to launch virtual devices

Setting Up Real Devices

  1. Go and purchase some smartphones (or use your own one) with SIM cards
  2. Install and register WhatsApp on them
  3. Enable Android developer mode — ask Google how to do it for your device model
  4. Attach it to your workstation with the USB data cable
  5. Install Android SDK
  6. Install and launch Appium

Setting Up Virtual Devices

  1. Install Docker and Docker-Compose
  2. Launch a virtual device with the help of this project

In short, you can create a file docker-compose.yml and launch it with docker-compose up -d to get a single virtual machine with a single virtual Samsung device up and running. You can see it in action by browsing to http://localhost:6080

Trending Bot Articles:

1. Case Study: Building Appointment Booking Chatbot

2. IBM Watson Assistant provides better intent classification than other commercial products according to published study

3. Testing Conversational AI

4. How intelligent and automated conversational systems are driving B2C revenue and growth.

version: "3"
image: budtmo/docker-android-x86-11.0
privileged: true
- "6080:6080"
- "4723:4723"
- DEVICE=Samsung Galaxy S8
- APPIUM=true
Samsung Device Emulator

Get a Twilio account or any other SMS provider, install WhatsApp on the virtual device and register it by SMS.

In one of the next Botium releases, the registration step will be automized by Botium Box as well.

Now that your devices are up and running and WhatsApp is available, lets see what else to prepare.

Connect Botium to WhatsApp

You need an installation of Botium Box for this purpose. Experienced DevOps engineers can give Botium Core a try, the free and open source automation library powering Botium Box and other Botium products.

Get your copy of Botium Box here

Connect Device Lab To Botium

In the Botium Box settings, register a new device provider.

Tell Botium about the Device

While for most device cloud providers the available devices can be listed automatically (by calling the device cloud listing APIs), this is not possible for your local Appium installation. Edit the file LOCALSELENIUM.json in the resources folder of Botium Box to tell Botium about the available devices:

"name": "Samsung Galaxy S8 Emulator",
"value": {
"type": "MOBILEAPP",
"capabilities": {
"appium:platformName": "Android"

Compose a Device Set

Botium Box groups the devices you want to run your tests on in device sets. Create a new device set for your Appium endpoint and select the Samsung Galaxy S8 Emulator (and maybe other devices as well if you connected them).

Configure Botium Connector for Appium

Register a new chatbot in Botium Box

  • As Connector/Chatbot Technology use WebdriverIO (Selenium or Appium)
  • As Automation Technology select Use Appium
  • As Webdriver Script choose Whatsapp

As an experienced Appium developer you might ask Where do I enter the Selenium CSS Selectors ? — with Botium you don’t have to do this as this is part of the Webdriver Script Whatsapp.

On the last step of the Quickstart Wizard, make sure to select your device set at the very bottom to start your tests.

Write Test Cases

You can now use the full power for BotiumScript to write your test cases. An easy one could look like this:

Welcome to the World Health Organization

Watch Botium Automating Whatsapp

When running a test case, Botium will now

  • open the WhatsApp app
  • select the contact representing the chatbot
  • cleanup the message history
  • send test case input from WhatsApp
  • receive WhatsApp output and compare with the test case
  • repeat the last two steps until the test case is ready


With Botium Box and Appium it is now possible to run automated end-2-end tests of conversational flow of your WhatsApp chatbot.

Don’t forget to give us your 👏 !

How to Automate Testing of Your WhatsApp Chatbot was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.





Deprecated: Function WP_Scripts::print_inline_script is deprecated since version 6.3.0! Use WP_Scripts::get_inline_script_data() or WP_Scripts::get_inline_script_tag() instead. in /www/eksidoio_182/public/wp-includes/functions.php on line 5453