Six weeks into running an AI support agent, I did the thing every founder eventually does at 11pm: I read every single 1-star CSAT review in one sitting. And a pattern jumped out that genuinely embarrassed me. Our agent wasn't hallucinating in the cartoonish way people warn you about. It was doing something quieter and worse. When a customer asked about our enterprise billing tier — which the agent had zero accurate data on — it would reason from our public pricing page and produce an answer that sounded completely authoritative. 'Enterprise typically starts around the $2,000/month range with annual commitment.' We don't even publish that number. It made it up from vibes, and customers made real decisions on it.
The maddening part is that a wildly wrong answer would have been safer. A customer reads 'our enterprise plan is $4 trillion' and laughs it off. They read a plausible, confident, specific number and they forward it to their procurement team. Confidence without grounding is the actual failure mode of support agents, and our original system prompt — which I'd written myself in about fifteen minutes — did nothing to prevent it.
Why I stopped writing system prompts by hand
My original prompt was a wall of friendly instructions: be helpful, be warm, answer the customer's question. There was no section that said 'here is precisely what you do not know.' There was no escalation protocol. There was a vague 'if you can't help, let them know' that the agent interpreted as 'try your absolute hardest to help anyway.' I needed structure, and I needed it written by something that thinks about scope boundaries for a living. So I ran my product through the Write a Bulletproof System Prompt for a Customer Support AI Agent prompt and handed the whole thing to Claude Opus 4.8, because Opus is genuinely good at honoring explicit structural sections and reasoning carefully about edge cases before it commits to output.
I was specific where it mattered. For support scope I wrote 'password reset, subscription management, and feature how-tos for the Starter and Growth plans only.' I listed enterprise billing, custom contracts, and SLA questions explicitly in the out-of-scope bucket. For escalation rules I wrote 'hand off any mention of enterprise pricing, any legal threat, any refund over $50, and any time the customer asks for a human.' Twenty minutes of writing inputs, versus the hours I'd lost to bad reviews.
The section I didn't know I needed
The output gave me a Knowledge and Scope section split into two bulleted lists — what it can help with, and what it explicitly cannot. That alone changed behavior. But the part that surprised me was the Refusal Policy. My instinct had always been to treat refusals as failures. The generated prompt reframed them: every refusal acknowledges the topic, redirects to what the agent CAN do, and hands off cleanly. The agent stopped saying 'Sorry, I can't help with that' (which customers hate) and started saying 'Enterprise pricing is handled by our billing team who can give you accurate numbers for your account — I'm connecting you now.'
- Enterprise billing questions now trigger a clean handoff instead of a confident invention — the agent literally cannot price something it was told it doesn't know
- The agent switched from saying 'I' to 'our team' for anything it escalates, and customers rated those interactions as noticeably more trustworthy
- CSAT climbed from 3.4 to 4.1 over the following month, and nearly all of the gain was in the categories where the bot used to overstep
- Escalation volume actually went UP at first — which was correct, because those were conversations it never should have handled alone
The honest caveat
The first generated version was a touch too rigid — it escalated a simple 'how much is the Growth plan' because the word 'pricing' appeared. I added one line to the escalation rules input ('Growth and Starter pricing is in scope; only enterprise/custom pricing escalates') and regenerated. Thirty seconds. That's the workflow now: generate, test on ten real tickets, tweak one input, regenerate. I pair this with my Write a Guardrail and Refusal Policy for an AI Agent prompt for anything customer-facing, because scope and safety are two halves of the same job.
If your agent has ever answered a question it had no business answering, the fix isn't a smarter model — it's a system prompt that draws the boundary explicitly. Grab the Write a Bulletproof System Prompt for a Customer Support AI Agent prompt on Prompt Dock, feed it your real scope and escalation rules, and run it on Claude Opus 4.8. The version of your agent that knows what it doesn't know is the one customers actually trust.