Most small businesses flourish because of the personal touch that the owners present to their customers. Large firms can maintain a good rapport with their customers by staffing the customer interfaces with personable and competent customer support people.
When a company completely replaces people with robots, unrecoverable errors multiply and customer trust vanishes.
If you search the internet for “AI Mistakes”, you will find hundreds of them and the reasons are many and they are not going away yet. The majority of the AI disasters are a result of adopting an AI system with no human backup. Management seeks to reduce payroll to improve the bottom line and automates financial transactions with AI systems scanning for signs of fraud.
Square has done this and the results are not pleasant. Armed with a list of conditions that indicate a high-risk customer, their AI systems acts swiftly to freeze any account that shows these signs of possible fraud: International financial transactions, Larger than normal transactions ($1000 or greater), and data entry errors on credit card transactions. The presence of any of these will trigger a freeze of funds for 90 days or more.
Those of you who operate an online business and have international customers have probably encountered all of these frequently and consider them a normal condition of any online business these days. If you are using any of the money processing services of the Square company, you may encounter their implacable AI system which will freeze your funds with no access to a human supervisor. More than 5000 customer complaints can attest to this situation.
This is a sign that management has bought into the idea that preventing fraud is more important than maintaining good relations with customers and a belief in the infallibility of AI systems. Automated systems rely on predictable data inputs and when this is missing we see self-driving car crashes and money processing systems that freeze your funds arbitrarily.
The internet abounds with solutions for the inflexibility and error prone operation of AI systems. One of the most applicable is to provide paths forward from failure, from the People + AI Guidebook. If you are using AI in your company, this quote should be posted on every wall.
Provide paths forward from failure. The trick isn’t to avoid failure, but to find it and make it just as user-centered as the rest of your product. No matter how hard you work to ensure a well-functioning system, AI is probabilistic by nature, and like all systems, will fail at some point. When this happens, the product needs to provide ways for the user to continue their task and to help the AI improve.
If You Are A Small Business Owner, Would You Trust A Robot To Service Your Customers? was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.