Human-in-the-loop for AI agents

What HITL is not

It is not "the model can ask a person if it wants to." That is an escape hatch nobody monitors.

It is not reviewing every token. That does not scale and teaches operators to rubber-stamp.

What works

Define triggers: hard rule fail, low field confidence, high-risk customer segment, amount over threshold.

Attach evidence: source pages, check results, prior decisions. Humans decide faster with context pre-loaded.

Log overrides with reasons. That dataset improves rules and evals more than more prompt tuning alone.

Queue design details

Every exception needs priority, owner role, SLA, and a resolution vocabulary: fix input, override with reason, send back, or escalate.

If the queue is a shared inbox with free-text notes, you have recreated email. Structured outcomes are what improve the system over time.

When to force human review

Hard rule failures, confidence below threshold, high-risk segments, amounts over a limit, and any write that cannot be easily reversed.

Publish that list. Operators and auditors should not guess when a human is required.

In practice

Map the workflow on a whiteboard before you open a framework: inputs, systems of record, humans, and irreversible writes. If that map is fuzzy, the agent will encode the fuzz.

Pick ten to fifty real historical cases as an eval set. Include the ugly ones. Run the agent offline against them until critical fields and hard rules are acceptable. Only then connect write tools.

Ship with a pause switch, a human queue, and a weekly review of override reasons. Promote repeated overrides into rules. That loop is how production systems improve—not another prompt brainstorm.

Common failure modes

• Treating a demo on clean samples as readiness for production volume.
• One shared service account with broad write access across systems.
• No owner for the exception queue, so failures pile up as noise.
• Changing prompts and models without regression gates on real cases.
• Measuring only model latency or thumbs-up, not completed-case cost and audit completeness.

What good looks like after ninety days

The first workflow is boring: stable override rate, known failure modes, operators who trust the queue. Config changes go through review. Traces answer "what happened to this case?" without archaeology.

At that point you can add a second document type or a second agent role. Expanding before the first path is boring is how programs stall with five half-built pilots.