Our open-source project gets around 15 new GitHub issues a week, and they fall into three buckets with boring regularity: bug reports, feature requests, and questions that really belong in Discussions. So I did what any reasonable maintainer does at the end of a long week — I decided to make a robot do it. The agent would read each new issue, apply a label, drop it in the right project board column, and for the question-shaped ones, post a friendly comment pointing the author to Discussions. Easy. Famous last words.
My first version was a six-step chain I designed off the top of my head. It passed all my test cases. It ran beautifully for three days. Then it posted, on a clearly-written bug report with a stack trace attached, the comment: 'This looks like a question — please post it in Discussions instead!' The author was, understandably, not delighted. I'd publicly told someone their crash report wasn't a real issue. The screenshot made the rounds in our Discord. Cool. Great. Love that for me.
The hole in my hand-rolled workflow
When I went back and looked at my chain, the flaw was obvious in hindsight: my classification step produced a label, but it had no concept of confidence and no branch for 'I'm not sure.' The failure path just... continued to the next step regardless. A 51%-confident 'this is a question' guess flowed straight into the 'post the redirect comment' step with the same authority as a 99% one. There was no gate. The agent was allowed to take an irreversible, public action on a coin flip.
What the workflow design added
I rewrote it using the Build a Chain-of-Steps Workflow for a Multi-Step Agent Task prompt and ran it on Grok 4.3, which I like for this because it's fast, direct, and doesn't pad the output with ceremony when I just want a clean decision graph. The workflow it produced had the gate I'd been missing wired right into the step definitions.
- The classification step now has an explicit decision point: confidence above 80% continues; below 80% applies a 'needs-human-triage' label and STOPS — no public action
- A verification step checks that the label applied actually matches the classification result; a mismatch triggers a correction instead of shipping the error
- The 'post a comment' step is flagged as a human-irreversible side effect, so it only fires on a confirmed high-confidence question, never on a guess
- Timeout handling: if the GitHub API lags, the agent retries once, then applies a 'triage-pending' label and exits cleanly instead of hanging or half-completing
The verification step I almost left out
When I first read the generated workflow, I nearly deleted the verification step as overkill. It seemed redundant: the agent applies a label, of course the label is applied, why check? I left it in out of laziness, and within a week it earned its keep. The GitHub API occasionally accepted a label-add call, returned success, and then... didn't actually apply the label, because of a race with another automation we were running. Without the verification step ('confirm the applied label matches the classification result'), the agent would have happily reported success on issues that were silently still unlabeled. The verification caught the mismatch and re-applied. The lesson: 'the tool returned 200' is not the same as 'the thing happened,' and a workflow that confuses the two will lie to you with a clear conscience.
Eight weeks later
The agent now handles about 70% of new issues with no human involvement at all. The other 30% land in the 'needs-human-triage' pile, and when I review them, they're genuinely ambiguous — a feature request phrased as a question, a bug that might be a usage error. The agent is correct to punt on those. The thing that makes this workflow trustworthy isn't that it's smart; it's that it knows the difference between 'confident enough to act' and 'guessing,' and it refuses to take a public, irreversible action on the second one.
What I'd tell anyone building their first action-taking agent: the design work is not about making the happy path work. The happy path always works in your demo. The design work is about every other path — the low-confidence guess, the slow API, the success-that-wasn't, the irreversible step. The Build a Chain-of-Steps Workflow for a Multi-Step Agent Task prompt forces you to name a failure path for every single step, and it was the discipline of that, more than any individual clever rule, that turned my embarrassing comment-bot into something I let run unattended.
If you're automating anything where a wrong step is visible or hard to undo, you need confidence gates and verification baked into the design, not bolted on after a public faceplant. Grab the Build a Chain-of-Steps Workflow for a Multi-Step Agent Task prompt on Prompt Dock and run your own task through it on Grok 4.3. I pair it with my Design a Tool Function Schema for an AI Agent prompt so each tool the workflow calls is described precisely enough that the agent picks the right one every time.