About six months ago we started building AI agents for internal workflows. First thing we shipped was a Slack bot pretty scoped, just handling repetitive team questions, routing requests, fetching status updates. Seemed simple enough.
Here’s what actually happened that I didn’t see coming.
The trust problem hit faster than expected. Within two weeks our team stopped double-checking outputs. Not because they were lazy because it worked well enough that they forgot it could be wrong. One bad answer sat unchallenged for two days before someone caught it. After that we deliberately built in a confirmation step here’s what I found, does this look right? before any action got finalized. Felt like a step backward but it fixed the problem.
Maintenance is the real job. Building the bot took maybe three weeks. We’ve now spent way more time than that just keeping it working prompts drifting, APIs changing, edge cases we never anticipated showing up in production. Nobody really talks about this part.
Context handling is harder than it looks. Slack conversations are messy. People mid-thread, no clear structure, references to things said three days ago. Getting the agent to handle that gracefully took a lot more iteration than the core functionality.
We’ve built more agents since then for different workflows each one had its own version of these same problems.
Anyone else building agents for internal tooling? Curious what broke first for you and how you handled it.
submitted by /u/Consistent-Arm-875
[link] [comments]