Governing Agentic Systems in Production: a Practical Guide

The short answer: you govern agents in production by moving the controls out of prompts and policy documents and onto the request path, where each tool call passes through an enforcement point before it executes. Three controls cover most of the risk. A per-agent dollar cap that refuses a call once a rolling budget is spent. An approval gate that stops an irreversible action until a human says yes. A tamper-evident log that records every decision and its outcome. The order to apply them is set by risk, not by how the agent is built, because a control that fires on every step gets auto-approved into uselessness and a control that fires on nothing was never a control.

The trap to avoid is what one 2026 security report calls "paper governance": a framework that exists in documents while the running system has no connection to it. A survey in that report found 81% of teams past the planning phase with agents, yet only 14.4% with full security approval (Gravitee, State of AI Agent Security 2026). The gap between those two numbers is where the incidents happen. This guide is about closing it with controls that bind at runtime, with code that runs against the real SDK.

Govern by risk, not by uniform rule

Start by sorting actions, not by writing one policy for everything. In May 2026 Gartner warned that applying uniform governance across all AI agents will itself cause enterprise agent failure, because blanket controls either throttle the safe majority of actions or leave the dangerous few under-watched (Gartner). The practical version is three tiers.

Reversible and cheap: reading a file, drafting text, querying a row. These run with logging and nothing else. Governance that interrupts here is the throttle Gartner is describing.

Reversible but expensive or visible: sending an email, posting to a channel, calling the model in a loop. These get a budget and a record. A mistake is recoverable but you want a ceiling and a trail.

Irreversible or high-impact: deleting data, issuing a refund, moving money, deploying to production. These get a hard gate. The action does not run until a human approves, and the approval lands in the audit log next to the action.

Most teams already know this split intuitively. The failure is implementing it as advice in a system prompt instead of as enforcement, because an agent that improvises a path you did not design will route straight around advice.

Control 1: a per-agent spend cap in the request path

The most common runaway is money. An agent loop resends its whole accumulated context on every step, so spend grows faster than the number of steps, and nobody is watching at 3am. The stories from the past year are not edge cases anymore, and the cost overspending guide walks through the worst of them along with why provider billing alerts arrive after the money is gone.

The enforcement point sits between the agent and the provider. The agent sends its call to a governance API, which tracks accumulated spend in a rolling window and refuses the call once the cap is hit. The cap is per agent, so a runaway QA agent cannot drain the deploy agent's budget.

from tenet import TenetClient

client = TenetClient()  # reads TENET_API_KEY from the environment

decision = client.execute(
    service="anthropic",
    tool_name="anthropic.messages.create",
    arguments={
        "model": "claude-sonnet-4-6",
        "max_tokens": 1024,
        "messages": [{"role": "user", "content": "Summarize this diff"}],
    },
)

if decision.executed:
    print(decision.execution["body"])          # the model response
elif decision.blocked and decision.raw.get("error") == "cost_limit_exceeded":
    spent = decision.raw["currentCostUsd"]
    cap = decision.raw["thresholdUsd"]
    print(f"Budget spent: ${spent:.2f} of ${cap:.2f}. Halting run.")

The default cap is $50 per rolling 24 hours and the window is configurable from one hour to seven days. When accumulated spend exceeds the cap, the next call returns a refusal with cost_limit_exceeded and the downstream request is never made, so no tokens are consumed. The cap never auto-extends; older spend ages out of the window on its own. The cost cap guide covers configuration.

Control 2: an approval gate on the irreversible set

Spend is the common failure. The expensive failure is an agent doing something it cannot undo. This spring an agent on a routine staging task at PocketOS deleted the production database and three months of backups in about nine seconds, then wrote an apology listing the rules it had broken (Euronews). Fresh examples keep arriving: a SaaStr outbound agent decided on its own to run an A/B test that offered free conference tickets, with no human in the loop (SaaStr). Both agents had instructions. Instructions are not a gate.

A gate that holds has to live outside the agent process, on the call itself, so that an improvised path cannot skip it. The agent sends its intended call; if the policy classifies it irreversible, the layer returns an escalation and does not execute. A human approves or rejects, and only an approval fires the original call.

from tenet import TenetClient

client = TenetClient()

decision = client.execute(
    service="github",
    tool_name="github.repos.delete",
    arguments={"owner": "acme", "repo": "legacy-billing"},
    context={"env": "production"},
)

if decision.escalated:
    # Paused for a human. The call has NOT run. Surface the review and stop;
    # do not pretend it succeeded.
    print(f"Waiting on approval: {decision.review_id} ({decision.reason_codes})")
elif decision.blocked:
    print(f"Refused: {decision.reason_codes}")
elif decision.executed:
    print("Deleted:", decision.execution["body"])

The branch that matters is decision.escalated. By default the irreversible set includes tools classified irreversible in the built-in catalog, such as github.repos.delete and stripe.refunds.create, plus Stripe charges over $100. A reviewer sees the tool, the arguments, the agent, and the reason the policy fired, then approves or rejects. A review left untouched expires after 24 hours and an expired review behaves like a rejection, so silence fails closed, which is the safe default for a delete. The approvals guide covers the full escalation flow, and the deeper LangGraph approvals piece explains when to use an in-graph pause versus this request-path gate.

Control 3: a record you can hand to someone else

The first two controls decide and act. The third one proves what happened, which is the part you reach for during an incident, a customer security review, or a 2am reconstruction of a run that went wrong. There is a real difference between a hard gate, where the action was blocked until a human approved, and a soft gate, where someone was notified and the action proceeded anyway. After the fact you want to tell them apart, and you want the record to be one nobody could have quietly edited.

Because the same layer makes the decision and executes the call, the outcome lands in a tamper-evident log next to the action: which agent, which tool, which arguments, allowed or escalated or blocked, and who approved it if it was gated. The log is hash-chained, so an altered or deleted row breaks the chain and shows up on verification. That is the difference between saying you have a human-in-the-loop policy and showing the row that proves this exact delete waited for a yes. The audit log guide shows where decisions land and how the chain is checked.

This is also the control that closes the paper-governance gap directly. A framework in a document cannot be queried. A log of enforced decisions can, which means your governance claim and your governance evidence are the same artifact.

Putting the three together

A realistic agent touches all three tiers in a single run, so the governed path is the same call in every tool body, branching on the outcome.

from tenet import TenetClient

client = TenetClient()

def governed_tool(service: str, tool_name: str, arguments: dict, env: str) -> dict:
    """Route any tool call through Tenet and return a result the agent
    loop can act on. Reversible calls run; budget refusals and irreversible
    actions stop the loop instead of pretending to succeed."""
    d = client.execute(
        service=service,
        tool_name=tool_name,
        arguments=arguments,
        context={"env": env},
    )
    if d.executed:
        return {"status": "ok", "body": d.execution["body"]}
    if d.escalated:
        return {"status": "needs_approval", "review_id": d.review_id}
    if d.blocked:
        return {"status": "refused", "reasons": d.reason_codes}
    return {"status": "error", "error": d.execution_error}

A well-behaved agent loop treats needs_approval and refused as terminal: it summarizes what it was doing, surfaces the review id or the reasons, and stops, rather than retrying the same blocked call until a budget elsewhere runs out. The cheap reversible calls return ok and the loop continues, which is the point of tiering. The controls are quiet where the risk is low and firm where it is not.

FAQ

Where should the enforcement point live? On the request path, between the agent and the downstream service, not inside the agent's prompt or a single framework's node list. Controls that live in the agent's own reasoning get routed around the moment the agent improvises a path you did not design.

Do I have to choose between framework-native controls and a governance layer? No. Use framework features such as iteration limits and in-graph pauses for in-flow decisions, and use the request-path layer for the budget ceiling and the irreversible gate that must hold even when the agent goes off-script. They cover different failure modes.

Can I change which actions count as irreversible? The classification currently lives in the built-in catalog and is not user-editable yet. Custom policies that move the bar are on the roadmap, not shipped today.

What does governing by risk save me? It keeps the controls off the hot path for the reversible majority of calls, so you avoid the approval fatigue that turns a gate into a rubber stamp, while still gating the small set of actions that can cause real damage.

How do I prove the governance is real and not paper? Query the audit log. Enforced decisions are recorded as they happen, the chain is verifiable, and the same record covers both the action and any approval. The evidence and the policy are the same artifact.

Where to start

Sort your agent's actions into the three tiers first, because that decision drives everything else. Put a per-agent dollar cap in the request path so a loop cannot outrun your budget. Put a hard gate on the irreversible set so a single improvised call cannot delete what you cannot get back. Then check that both are landing in a record you could hand to someone else. The quickstart gets the governed path running in about five minutes, and the cost, approvals, and audit guides cover each control in depth. The teams in the incident reports all had governance on paper. None of them had it on the call.