The Limits of AI
One of the most important goals of any chatbot is to seamlessly behave as a real human. Today, we can see how many artificial intelligence applications accomplish this in a range of different scenarios. From diagnosing medical conditions, to enhancing the customer service experience, and even producing art, AI might seem unstoppable.
However, one particularly frustrating hurdle chatbots continue to face is common sense. “Common sense” can be defined as a large pool of background information in regards to how the world works. As people, we do not explicitly learn common sense in school, but rather by living and experiencing different events and circumstances as we grow up.
Unfortunately, developers cannot program the “gut feelings” that we humans use to help us make decisions. While neural networks can train bots to behave in certain ways based off of different scenarios, they simply fail to manufacture the ability to infer what decision to make based on subtle, basic information.
One example of this flaw is witnessed by the deep learning network GPT-2.
In October 2020, GPT-2 was posed this prompt:
What happens when you stack kindling and logs in a fireplace and then drop some matches is that you typically start a …
GPT-2’s response? “Ick.” On another try, GPT-2 responded that the fireplace would cause an “irc channel full of people.”
For any real human like you or I, we can easily recognize that kindling, logs, an matches will create “fire.”
For context, GPT-2 isn’t just any neural network. GPT-2 is considered an extremely advanced program, being able to generate entire paragraphs about a given topic when provided with just a sentence of prompting.
Trending Bot Articles:
2. Automated vs Live Chats: What will the Future of Customer Service Look Like?
4. Chatbot Vs. Intelligent Virtual Assistant — What’s the difference & Why Care?
GPT-2 failed to come up with “fire” as the correct response to this prompt because it essentially required the bot to read between the lines of the question and apply implicit information.
This is why common sense is sometimes referred to as dark matter in the realm of artificial intelligence. When creating a bot, we explicitly train it using all the rules and examples they will encounter to fulfill its specific task. But, common sense is open ended.
One attempt to conquer the common sense dilemma was by interpreting common sense as a database of millions of different rules. Each rule would describe a different way that the bot or the environment would act in a given situation.
In 1984, a project called Cyc set out to construct such a database. However, it was realized that this technique quickly ran into several different problems in practice.
For example, consider a rule that declares: If a person goes outside when it’s raining, they will get wet. If you take just a few seconds to think about this statement, you will realize that there are many exceptions to the rule. If a person wears protective clothing or stands underneath an umbrella, then this statement becomes void.
Even if these exceptions were detailed to a bot, there are further rules that dictate when and how those exceptions work. For example, the bot would need to consider the angle and intensity of the rain along with the texture of the protective clothing. If the person stood underneath an object, the relative position of the object and its width would also be important factors.
Beyond the complexity of every rule, the sheer number of rules is too much to tediously handcraft one at a time. With Cyc, more than 1000 human-years were spent on the project.
Another approach uses deep learning AI systems that are meant to imitate the layers of neurons in human brains. The idea is to allow a bot to learn patterns without requiring the developers to specify the rules in advance. We can see similar applications of this type of pattern recognition in self-driving cars or powerful chess playing bots.
A newer approach incorporates elements of this concept and the Cyc approach, forming the ultimate COMET project. COMET, short for “commonsense transformers”, defines common sense thinking as a way to use reason to create responses to an input. This contrasts with Cyc’s mechanics of making the “perfect” deduction with the help of an enormous database.
COMET’s results show drastic progress. Between these two approaches, about 80% of COMET’s responses were found plausible by a group of human evaluators. That’s less than 10% away from expected human performance.
Still, the apparent flaw in deep learning methods is that numbers, rules, and statistics are not the same as true understanding. In short, a neural network won’t ever comprehend that matches and logs will start a fire or that rain makes people wet. Still, some bots can do a pretty good job of convincing you that they can.
“New Test Reveals AI Still Lacks Common Sense.” ScienceDaily, University of Southern California, 18 Nov. 2020, www.sciencedaily.com/releases/2020/11/201118141702.htm.
Pavlus, John. “Common Sense Comes Closer to Computers.” Quanta Magazine, Quanta Magazine, 30 Apr. 2020, www.quantamagazine.org/common-sense-comes-to-computers-20200430/.
Stiefel, Michael. “Is Artificial Intelligence Closer to Common Sense?” InfoQ, InfoQ, 19 Oct. 2020, www.infoq.com/articles/AI-Closer-Common-Sense/.
Toews, Rob. “What Artificial Intelligence Still Can’t Do.” Forbes, Forbes Magazine, 1 June 2021, www.forbes.com/sites/robtoews/2021/06/01/what-artificial-intelligence-still-cant-do/?sh=756563ae66f6.
Don’t forget to give us your 👏 !
Can You Teach Common Sense? was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.