Human in the loop AI agents: designing approvals that hold up

Approvals are the most effective control for AI agents, and the easiest to get wrong. Which tool calls need one, why each approval should be single-use, how timeouts behave, and what auditors will ask for.

VeloPhex Product

Product and automation practice

Human in the loop AI agents pause before risky actions and wait for a named person to approve that exact call. Good approval design decides which tool calls need a person, binds each approval to one call and its arguments, makes expiry mean "no", and records who asked, who decided and what ran, so auditors can follow it.

Approvals are the control most teams reach for first when they put an AI agent near a real system. That instinct is right. A model that can be talked into the wrong action by a cleverly worded email is much less dangerous when the action needs a person's click.

It is also the control most often built badly. An approval that is shown too often gets rubber-stamped. One that can be reused becomes a standing permission. One that never expires sits in a queue until someone approves it on a Friday afternoon without context. This post is about getting the details right. Most of it applies to any agent framework. We show an approval gate in LangGraph, and at the end, how the VeloPhex beta implements it.

Human in the loop vs human on the loop

The two phrases sound alike and describe different controls.

• Human in the loop: the agent cannot complete a specific action until a person acts. The person is a required step. Nothing happens without them.
• Human on the loop: the agent acts on its own while a person watches, with the ability to stop runs, reverse actions or change the rules. The person supervises; they are not a step.

Human on the loop scales better, because nobody has to click for every action. It is only safe when actions are reversible, monitoring is good enough to spot a problem quickly, and stopping is easy. Human in the loop costs reviewer time, but it is the right control where a single wrong action is expensive. Most production agents use both: on-the-loop supervision for the run as a whole, and in-the-loop approval on the few tools that can do real damage.

Four human-in-the-loop patterns

"Human in the loop" covers more than approve-or-reject. Four patterns cover most needs:

The rest of this post is mostly about the first pattern, because it is the one that stands between an agent and a real effect. The other three matter just as much for quality, and they are often cheaper: a review queue or an escalation path holds no running process open.

Approve actions, not agents

The first design decision is the unit of approval. There are three common choices:

Approving the agent belongs in your release process (see treating automation as software). Approving individual actions belongs at run time. Plan approval sounds efficient, but agents re-plan after every tool result, and a plan approved at step one may not resemble what happens at step six.

Which tool calls need approval

Not every call deserves a person's attention. If every lookup needs a click, approvers stop reading. A useful rule: put an approval in front of any call where a mistake would need an incident report. In practice, sort each tool along four questions:

1. Can it be undone? Reading a policy document: yes, there is nothing to undo. Sending a payment, deleting a record or emailing a customer: no, or only with effort.
2. Who sees the effect? Effects visible only inside the team are cheaper to get wrong than effects a customer, supplier or regulator sees.
3. Does it move money or change access? Payments, refunds, credit limits, role grants and password resets always deserve a second look.
4. Does data leave the organization? Any call that sends content to an external endpoint is a possible exfiltration path, especially if the content came from an untrusted source.

That gives a simple default policy:

Two patterns reduce approval volume without weakening control. First, move thresholds into code: if refunds under EUR 50 are routine, let a deterministic workflow handle them, and give the agent only a tool that queues larger ones for review. Second, split tools: a "create draft" tool needs no approval, while "send" does.

Single-use approvals, bound to arguments

Once a call needs approval, the approval should cover exactly that call. That means three properties:

• Bound to the arguments. The approver sees the arguments, and the approval is valid only for those arguments. If the agent changes the amount from 980 to 9,800 after approval, the approval does not match.
• Used once. Approving a call does not approve the next call to the same tool. An agent in a loop has to ask again.
• Enforced outside the agent. If the agent's own code decides whether an approval exists, a prompt injection that convinces the agent to skip the check also skips the control. The system that executes the tool must refuse calls without a matching approval.

The last point is the one most homegrown implementations miss. An approval step implemented as "the agent calls ask_human() and then proceeds" is a suggestion, not a control.

Timeouts: expiry must mean no

Every approval request needs a lifetime. The questions to settle:

• How long? Long enough for a real approver to respond during working hours, short enough that the context is still true. An invoice approval that waits overnight may approve an invoice that has since been paid by another route.
• What does expiry do? It must refuse. The call is not made, and the agent is told so. Any design where silence counts as consent will be exploited by whoever learns that approvers are slow.
• What happens to the run? Usually it continues without the action and reports that it could not complete, or it ends with a clear status. Either way, a person can see what was not done.
• What if the run ends first? Open approvals for a run that has finished should expire with it. An approval that outlives its run is a loaded tool waiting for a caller.

Blocking approvals suit decisions that take minutes. For decisions that take days, such as a contract exception, use a human task in a workflow instead, where waiting is the design and nothing holds a running process open.

Reviewer fatigue and rubber-stamping

The quiet failure of every approval system is the approver who stops reading. It happens for predictable reasons: too many requests, requests that all look the same, and a history of the agent being right. After the fiftieth correct refund, the fifty-first gets a click, not a look. A rubber-stamped approval is worse than none, because it produces evidence of a control that did not happen.

Watch for the signals:

• The approval rate drifts toward 100%. Some agents are that good; more often, nobody is checking.
• Decision times collapse to a few seconds, too short to read the arguments.
• Approvals arrive in bursts, a whole queue cleared in a minute at the end of the day.

And design against it:

• Cut volume first. Move routine cases below a threshold into deterministic rules, so the requests that remain deserve attention.
• Show what is unusual. Highlight the amount against the supplier's normal range, a new bank account or a first-time recipient, rather than showing every request the same way.
• Sample a second review. Have a second person re-check a small random share of approved requests, and treat disagreement as a signal about the process, not the person.
• Spread the load. Rotate approvers and cap how many requests one person handles in a sitting.

An approval gate in LangGraph

If you build agents with LangGraph, the interrupt function is the building block for the approve-before-acting pattern. A node calls interrupt() with a payload for the approver, the graph pauses and saves its state through a checkpointer, and a later call with Command(resume=...) on the same thread_id continues the run with the person's decision.

from typing import Literal, TypedDict

from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import END, START, StateGraph
from langgraph.types import Command, interrupt


class State(TypedDict):
    invoice_id: str
    amount: float
    status: str


def approval_gate(state: State) -> Command[Literal["post_invoice", "__end__"]]:
    decision = interrupt({
        "action": "post_invoice",
        "invoice_id": state["invoice_id"],
        "amount": state["amount"],
    })
    if decision.get("approved"):
        return Command(goto="post_invoice")
    return Command(goto=END, update={"status": "rejected"})


def post_invoice(state: State) -> dict:
    # Call the ERP here. This node runs only after an approval.
    return {"status": "posted"}


builder = StateGraph(State)
builder.add_node("approval_gate", approval_gate)
builder.add_node("post_invoice", post_invoice)
builder.add_edge(START, "approval_gate")
builder.add_edge("post_invoice", END)
graph = builder.compile(checkpointer=InMemorySaver())  # use a durable checkpointer in production

config = {"configurable": {"thread_id": "INV-2026-0416"}}
paused = graph.invoke({"invoice_id": "INV-2026-0416", "amount": 980.0, "status": "pending"}, config)
print(paused["__interrupt__"])  # show this payload to the approver

result = graph.invoke(Command(resume={"approved": True}), config)
print(result["status"])  # "posted"

Three details from the LangGraph docs matter here. When the run resumes, the node restarts from its beginning, so keep anything with side effects after the interrupt() call or in a separate node, as post_invoice is. Do not wrap interrupt() in a broad try/except, because the pause works by raising an exception. And the gate is only as strong as the code around it: the graph decides that post_invoice cannot run without passing the gate, but binding the approval to these exact arguments, recording who approved, and expiring stale requests are yours to build.

The VeloPhex beta equivalent. In VeloPhex Managed Agents you do not write the gate yourself. You mark the tool with approval: before_call in the agent definition (the "Approve each call" switch in the agent builder), and Orchestrator pauses the run before that tool is called, shows the approver the arguments, and refuses the call unless a matching, single-use approval exists. The next section walks through it.

What auditors need from an approval

When an auditor samples an agent action, they will want to reconstruct the decision. Plan for these records from day one (our guide to audit trails auditors actually use goes deeper):

Two details often get missed. Rejections and expiries are evidence too: a control that never rejects anything is a control an auditor will question. And the link between approval and execution matters: "Alice approved a payment" is weaker evidence than "Alice approved this payment and it was executed exactly once, two seconds later, by run 7f3c."

How VeloPhex handles approvals (beta)

Approvals are part of VeloPhex Managed Agents, which is in beta on a preview Orchestrator and open to Design Partners. Here is how the pieces above map to it.

Marking a tool. In the agent builder, each tool has an "Approve each call" switch, which sets approval: before_call in the agent definition. In this example, the vector store lookup runs freely while the invoice-processing robot needs approval.

The run pauses. When the agent decides to call an approval-gated tool, the run shows "Approval required" and waits. Model calls, tokens and cost so far are visible on the run.

A person decides. The Approvals page lists pending requests with the tool, the exact arguments and an expiry time. The approver approves or rejects that one call.

The run continues. The approval appears as its own step in the run trace, followed by the tool call it allowed.

Under the hood, in the beta:

• Orchestrator enforces approvals, not the agent runtime. A gated call without a matching approval is refused with approval_required.
• Each approval is single-use. It is consumed by exactly one call. For HTTP and MCP tools, it is bound to that call's exact arguments. For process (robot) tools, it is currently bound to the tool rather than the arguments, because the robot's package download does not yet carry an inputs hash; binding those to arguments is planned.
• Approvals expire. A request lives at most 15 minutes and never past the job's deadline, and open requests expire when the run ends. A rejected or expired approval means the call is not made, and the agent is told so.
• It is all recorded. Requests, approvals, rejections and consumption are written to the append-only audit log, and each approval records who decided and when.

A short checklist

• Every tool in every agent has an explicit approval decision, with a reason.
• Approvals cover one call and its arguments, and are enforced by the system that runs the tool.
• Expiry refuses, and its lifetime matches how fast approvers really respond.
• Approval requests show enough context to decide without opening three other systems.
• Approval, rejection, expiry and execution are all in the audit trail, linked to each other.
• Approval volume is reviewed monthly; if approvers approve everything, move routine cases into deterministic rules.

Approvals are one layer. Budgets, grants and prompt-injection defences are the others; we cover them in AI agent guardrails. For the bigger picture, see our overview of agentic automation.

Want to try approvals on a real process? Apply to the Design Partner program.

Frequently asked questions

What does human in the loop mean for AI agents?

It means a person must approve specific actions before an agent performs them. In a well-designed system, the agent can reason, read and draft freely, but calls that change records, move money or send data outside the organization pause until a named person approves that exact call. The platform, not the agent, enforces the pause.

Which agent tool calls should require approval?

Calls that are hard to reverse, visible to customers or partners, move money, change access, or send data outside the organization. Read-only lookups and retrieval usually do not need approval; they are better controlled with scopes and grants. If a wrong call would need an incident report, put an approval in front of it.

Why should an approval be single-use?

An approval that can be reused becomes a standing permission. If an agent can call an approved tool again, with different arguments or in a later loop, the person who approved it has authorized something they never saw. A single-use approval covers one call, with the arguments shown to the approver, and is used up when that call runs.

What happens if nobody responds to an approval request?

The request should expire, and expiry should mean no. The call is not made, the agent is told it was not approved, and the run either finishes without that action or fails visibly. Silent approval on timeout defeats the purpose. Set expiry to match how quickly a real approver can respond.

Does VeloPhex support approvals for AI agents?

Yes, in beta. In VeloPhex Managed Agents, a tool marked approval before_call pauses the run until a person approves that one call. Orchestrator enforces it server-side, each approval is single-use, open approvals expire with the run, and requests and decisions are recorded in the audit log. It runs on a preview Orchestrator, open to Design Partners.

Human in the loop vs human on the loop

Four human-in-the-loop patterns

Approve actions, not agents

Which tool calls need approval

Single-use approvals, bound to arguments

Timeouts: expiry must mean no

Reviewer fatigue and rubber-stamping

An approval gate in LangGraph

What auditors need from an approval

How VeloPhex handles approvals (beta)

A short checklist

Frequently asked questions

What does human in the loop mean for AI agents?

Which agent tool calls should require approval?

Why should an approval be single-use?

What happens if nobody responds to an approval request?

Does VeloPhex support approvals for AI agents?

8 UiPath alternatives for 2026 (and which suit Python teams)

If your automation team writes Python, the right platform looks different. A fair look at UiPath and its alternatives, including where VeloPhex fits and where it does not yet.

Product

AI agent guardrails: budgets, grants and prompt-injection defence

Guardrails are not one filter on the model's output. They are a set of limits around the whole run: budgets, grants, schema validation, untrusted-content fencing, a tool-call ledger and evaluations before publish.

Architecture

VeloPhex Engineering

MCP enterprise security: Model Context Protocol in production

The Model Context Protocol makes it easy to give AI agents tools. That is exactly why it needs controls. What MCP is, the four risks that matter, and the controls that address them.

AI & Agents

VeloPhex Security

RPA platform buyer's guide 2026: criteria, scoring and vendors