Agents need harnesses, not bigger keys
The part I keep tripping over in agentic ops is the handoff into systems that remember mistakes.
The agent has found the migration, patched the config, read enough logs to make a plausible case, and now it wants to run the thing. Maybe that means terraform apply. Maybe it means a privileged playbook. Maybe it means touching the kube context you really do not want to touch at the end of a long debugging thread.
The human gets a bad job: decide whether the plan is still current, the inventory is stale, the rollback is real, and the health check measures the system rather than the host the agent happened to query. The approval dialog usually does not know any of that.
Calling that autonomy feels generous. Mostly it is tab-juggling with a faster typist.
Treating the agent like a new employee helps only up to a point. Give it an identity, attach permissions, and you still have the same question at the dangerous moment: what is this operation about to change, what evidence justified it, and what happens if the answer is wrong?
I would put the control point on the work item. The record needs the approved target, reviewed artifact, minted lease, and recovery state. Pull requests, terminal sessions, deploys, and rollbacks hang off that record. A second agent should be able to continue from it without believing the first agent’s summary.
By lead agent, I mean the one carrying the task: holding the goal, talking to the human, delegating to helper agents, and deciding what to try next. It can coordinate, summarize, and propose. It should not be the source of truth or the evidence source.
Call the thing in the middle a harness.
A harness sits between agents and real systems and turns intent into controlled execution. If approval covered plan digest abc and the current plan is def, the run dies there. If the operation needs AWS, the harness mints a short Vault lease for the one target instead of handing over a cloud key. The transcript and postcheck land next to the approval.
The bad version is familiar: a GitHub comment approves “the plan,” CI has one digest, Vault issued a lease for a target, Ansible skipped three tasks in check mode, and the postcheck lives in monitoring. Nothing proves those facts belong to the same operation.12
Start with an operation contract
When people hear “harness,” they often hear “another platform.” That is not the part I would build first.
Do not rebuild Vault, OPA, CI, session recording, or Terraform. Pick the repeated operation that creates real babysitting and give it a record.
For restart_service_with_health_check, the first record can be plain: service, host, drain result, health check, lease path. The executor can be small. The important bit is that approval binds to a specific artifact, not to a friendly paragraph from the agent.
I care less whether that layer comes from a vendor product or a 200-line internal service. I care whether it rejects stale artifacts, issues only the credential needed for the run, and leaves enough recovery state for the next person.
Keep authority outside the model
Agents are usually trying to help. Treating them as hostile all the time makes them useless.
But any agent that reads untrusted content can be hijacked for the duration of a task. It does not have to be dramatic. A README says to ignore the test failure. A support ticket looks like an instruction. A log line contains a pasted production command. OWASP putting prompt injection at the top of its 2025 LLM risk list is less interesting as a headline than as a design constraint.3
The failure is not theoretical. Researchers have demonstrated support-ticket prompt injection leading agents with database tools to expose private tables.4
Text the agent reads should not widen the credential, tool, or approval it receives.
Let agents inspect, test, draft, and propose. Do not let them mint their own production reach.
Put dangerous systems behind named operations
apply_network_config_with_revert(host, config_digest, timeout) should refuse a stale digest, arm rollback before touching the interface, and prove reachability through the management path before it writes success. If the health check is just curl localhost, it has not tested the dangerous part.
Behind the verb there may be serious power. The failure mode is the escape hatch with a nicer label: an approved operation that eventually calls run_shell, execute_sql, run_arbitrary_playbook, or use_admin_browser with agent-written arguments.
Break-glass needs escape hatches. Normal autonomy should not depend on them. If every hard case routes through a god-verb, the harness is mostly theater.
Gate on consequence
Evidence is the input, not the verdict. A Terraform apply, an Ansible rollout, and a live debugging session should not pass through the same approval shape.
Ansible is the uncomfortable middle case. Check mode is useful, but only modules that support it report what they would change; unsupported modules report nothing and do nothing in check mode.5 If forty tasks exist and twelve were simulated, the approval should not say “apply the playbook.” That run starts as one canary and earns the next host only if external health agrees.
Terraform is cleaner. If I have the plan digest, I bind approval to the digest; if the commit, target, inventory, or plan changes, the old approval expires.6
Live debugging goes the other direction. There is no dry-run for “debug why this host is broken.” The session changes as it proceeds, and the first bad network command may cut off access. Before that run starts, I want the backup, time box, transcript, scoped credential, and out-of-band path named.
Netplan has a native example. netplan try applies a network configuration and reverts after a timeout unless the change is confirmed; the documented default timeout is 120 seconds.7 That gives you a recovery path.
The run earns expansion; it does not get it up front.
Prediction is strong and blast radius is small: bind approval to the exact plan.
In an incident review, I want the record to show what was approved exactly, what was bounded, and where the system stopped. If the system cannot say that, the approval is just a person lending their account to a guess.
Reversible is not harmless
Rollback restores configuration. It does not erase the outage.
An agent can restart the right service at the wrong moment. A migration can roll back cleanly after saturating a queue. A deployment can revert the code and still leave caches, replicas, or dependent jobs in a bad state.
Some effects survive the revert. By then, users have seen errors, queues may have duplicated jobs, and the email provider already has the messages. Payment side effects, data drift, cache poisoning, and downstream retries can all outlive a clean rollback. At that point you are no longer restoring a system shape. You are compensating for damage.
The operation needs a disruption budget, not just a rollback button.
For a restart verb, the disruption budget might say: one host drained at a time, queue depth under 500, five minutes of clean health before the next batch. For a migration, it might say: lock time under a threshold, fallback reads still working, rollback allowed only before the second write phase.
Those values should drive the executor. If queue depth crosses the limit, the run stops before the next host, not after the agent writes an apologetic summary.
The approval should name the disruption envelope. If the action needs more than that envelope allowed, it needs a new approval.
Local action: record the operation and let the token pass.
The proposer cannot be the evidence source
The weakest version of this system asks the agent to attach its own safety memo.
If the same agent recommends an action and writes the safety case for it, the human is reviewing a narrative controlled by the party asking for approval.
The bad packet often looks fine. The agent says rollback is safe, but the migration tool produced no down migration. It says the blast radius is one host, but inventory was stale. It links a health check from the host it just reconfigured. That is a story, not evidence.
I would rather see the migration tool’s output than a well-written paragraph from the agent.8
Probes are fallible. A stale inventory record fails in a different shape than an agent-written safety memo, and that difference is what makes the approval worth anything.
A chat transcript is too thin for provenance. The run record has to connect the request, measured evidence, minted capability, execution, and changed state.
Sandboxes still have a blast radius
Before an agent knows the fix, it needs room to poke at the broken state. Production should not be that room.
Run uncertain work where damage is bounded. Isolate it, then check what still leaks.9
“Sandbox” is weaker than “safe.” The failures are ordinary: a scratch clone carries a production .env; an ephemeral VM can reach a metadata service; a copied database still contains sensitive records; a test registry token can publish a real package or sign an artifact.
Before I call the sandbox safe, I want plain answers: can it mutate production, read secrets, reach metadata, or publish and sign something that leaves the sandbox?
The agent should leave behind something less fragile than shell history: a pull request with a test, a monitor, a rollback note, or a named operation. GitOps is excellent once desired state is known. It is clumsy while discovering what desired state should be.
The catalog grows from that loop: bounded exploration, one reviewed operation, then another. It is slower than letting the agent keep the shell, but the next run starts from a reviewed operation instead of an open terminal.
Partial failure is where approvals get stale
The playbook changed three hosts and died on the fourth. The local health check passed, the queue broke ten minutes later, and now the rollback only describes half the system.
At that point the original approval no longer describes reality. Recovery starts from a different state.
The harness needs to know the last completed checkpoint, what was mutated, which idempotency key is still live, and what state recovery starts from. Durable workflow systems can preserve state and resume execution, but they do not decide what is safe. That belongs in the operation design.
Crossing an abort threshold should create a recovery work item, not invite the agent to improvise in the same thread. Capture current state, stop further expansion, name the owner, and make the next action explicit.
The approval graph should follow irreversibility boundaries. A restart retry and a compensating data fix do not belong under the same old click.
Two clocks
Moving authority out of agents does not remove authority. It concentrates it in the harness and operation catalog.
Running restart_service_with_health_check can be fast. Changing that verb so it accepts a glob, skips drain, or targets a new service class is a production change. Every future run inherits that mistake.
A new verb is not a helper function. It needs an owner, tests for ugly parameter values, logs that survive a bad run, and a way to remove it.
The service owner should review catalog edits the same way they review a deploy path or admin role. A bad verb is a control-plane bug, not a typo.
Start with one operation
In an ops-heavy system, the first candidate might be deploy-time restarts: restart_service_with_health_check before deploys. It forces the first useful decision: who can restart what, with which lease, and what stops the next host.
After a few verbs exist, incident reviews get more useful. A failed run either tightens an existing operation or justifies a new one.
The approval I want to see is boring: restart nginx on host X, using plan Y, with rollback armed and queue depth below Z. A human can answer that. They are no longer being asked to lend the agent a cloud account and hope the summary is true.