i run an ai chatbot for business websites. last month i did something i’d been putting off for almost a year. i exported every conversation from the past 30 days across {N} tenants and read all of them. by hand.
the goal was to answer one specific question: when people talk to a customer-facing chatbot, what do they actually ask?
i’d been telling tenants the standard pitch: ai handles the long tail of customer questions, your support team handles the rare edge cases. it sounded right. i wasn’t sure it actually was.
here’s what i did:
step 1: pulled every user message from the last 30 days. {N} conversations, around {N} user messages.
step 2: stripped out the throwaway stuff. greetings, “thanks”, “ok bye”, angry venting, accidental sends. left with the actual questions: around {N}.
step 3: categorized by intent, not by wording. “what’s your refund policy” and “can i get my money back” go in the same bucket. “what time do you open” and “are you open today” same bucket.
step 4: counted.
what i found surprised me even though it shouldn’t have:
– the top 12 question types covered 73% of all messages
– the top 5 covered 51%
– the top 1 covered 19%
– the long tail (everything outside the top 50) was 11%
the long tail everyone worries about is real but it’s small. the head is way bigger than i’d assumed.
the 12 question types, in order, looked roughly like this:
- pricing / cost / quotes
- hours / availability / location
- shipping / delivery times
- product specs / does it do X
- refund / return policy
- account / login issues
- how to cancel / pause subscription
- how to contact a human
- discount / promo / coupon questions
- billing / charge questions
- integration questions (“does it work with X”)
- trial / demo requests
what i think this means for anyone running a customer-facing business:
a chatbot trained on 12 well-written canonical answers covers most of your inbound. the rest can route to humans. you don’t need a 200-page knowledge base for the bot to be useful. you need 12 short, confident, accurate answers and a fallback that doesn’t lie.
second thing, and this is the part i think about now: the questions in your top 12 are also your marketing problems. if 19% of incoming chats are asking about pricing, your pricing page is probably broken. if 8% are asking how to cancel, your cancellation flow is buried. the chatbot data is a product audit.
you don’t need a chatbot to do this exercise. pull 100 emails from your support inbox or 100 messages from your contact form. categorize by intent. you’ll find your top 12 too. probably less time than you think.
submitted by /u/FinanceSenior9771
[link] [comments]