The Agent Runtime Has to Become Inspectable

The message arrives at 9:12 on a Monday morning, and the first strange thing about it is that nothing is broken. The dashboards are green. The agent ran the job it was given, recorded every step along the way, and finished comfortably inside its time budget. No service is down, no error fired, and by every measure the engineer has been trained for fifteen years to watch, the system is healthy.

The message is from the customer's compliance office, and it asks a question the dashboard has no field for: who approved this?

She opens the trace, because that is what a decade of tooling has taught her to do, and it is all there and all useless. Model call, tool call, retrieval, another model call, the action, the confirmation. She can see exactly what the agent did. She cannot see whether it was allowed to do it. The run finished hours ago, and whatever the agent believed at the moment it acted, whatever rule it thought it was working under, whatever it should have needed a human to sign off on, none of it was written down anywhere. It is simply gone.

We have done this before

The software industry learned this lesson during the move to cloud infrastructure.

For years, debugging meant going to the machine where the problem happened and looking around. That made sense when servers were durable things: named, configured by hand, and expected to stay in place. Then the machines stopped staying in place. A request might pass through containers that existed for minutes, services that autoscaled under load, and instances that were gone before anyone knew there had been a problem. By the time an engineer went looking, the box that served the bad request had already been recycled.

The fix was not better luck or more careful engineers. It was a change in where the information lived. If you could not go to the system after the fact and ask what happened, the system had to tell you while it happened. Google published its Dapper paper in 2010, describing how it threaded a trace through every request crossing thousands of machines. Structured logs, metrics, and distributed tracing became standard equipment. The whole discipline got a name, site reliability engineering, which Google had been building since 2003 and codified in a 2016 book that a generation of engineers read like scripture.

The lesson underneath all of it was simple and expensive to learn. Inspectability is not something you add to a system after something goes wrong. Either the system was built to emit the evidence, or the evidence does not exist. Charity Majors, who went on to co-found the observability company Honeycomb, has described the moment she understood this: at Parse, the mobile-backend startup where she was an early engineer before Facebook acquired it, she realized they had built a system that was "basically undebuggable, by some of the best engineers in the world doing all of the right things." The people were good; the discipline was missing.

An agent run is ephemeral in exactly the way those servers were. The context window is gone the moment the run ends. The intermediate reasoning, the state the agent held in its head, the version of the world it was looking at when it decided to act, all of it evaporates. You cannot SSH into a finished agent run any more than you could SSH into a container that scaled away an hour ago. And most agent stacks today are sitting where the industry sat in 2010: they have logs that tell you the job ran and uptime checks that tell you it finished. That is the monitoring stage, and it records only that something happened.

The field is starting to build it in

You can see the same lesson starting to show up in agent research.

OpenAI's deployment-simulation work is a good example. Instead of asking only whether a model can clear a benchmark, it replays real past user conversations in a privacy-preserving way, removes the original assistant answer, and has the candidate model answer instead. The point is not to predict every failure. It is to test the candidate model against the kinds of conversations users actually had, rather than only against prompts researchers wrote because they already knew what they wanted to measure.

OpenAI is still describing a pre-release test. The model is not acting in the world yet; it is being run through yesterday's traffic before tomorrow's launch. Once agents start taking actions, the timing problem gets sharper. If the system does not preserve evidence while the work is happening, there may be nothing useful to reconstruct afterward. A robotics paper pairs a policy that proposes actions with a separate verifier that checks them at the moment of execution, then feeds the verified runs back into training. A reproducibility benchmark treats the reproduction problems people file as GitHub issues as a naturally occurring audit trail, evidence drawn from the work itself rather than from a model's report on its own work. The common thread is that the output is no longer enough. The system has to leave behind evidence of how it got there.

The thing SRE never had to carry

Everything the last generation built was in service of reliability. Observability answered two questions, what happened and why did it break, and those questions were enough because a web service does not act on anyone's behalf. It serves a request. When it fails, the failure is the problem, and the trace exists to help you find the cause.

Return to the Monday morning message. The dashboards were green because, in the narrow reliability sense, nothing had failed. The agent completed the job, stayed inside its time budget, and left a record of the steps it took. But the compliance office was not asking whether the run completed. It was asking whether the run was allowed.

That is the question the trace could not answer. Who approved the action? What policy was the agent operating under? Was the decision reversible? The system had recorded activity, but not authority. That is a different kind of evidence, and it sits on top of everything observability already learned to capture. A useful agent record has to separate what the agent observed from what it inferred, what it was authorized to do from what it actually did, and what it claimed happened from what can be confirmed. Authority is the layer the reliability world never had to think about, because reliability never handed software delegated permission to do things on its own.

That is where ordinary observability runs out. A trace can help debug a failure, but it cannot prove an agent stayed inside its scope unless the scope was recorded too. We have run into the same wall from the other direction in our own work on agent coordination. The same thing happens when agents hand work to each other. A summary can smooth over the uncertainty that produced it, so the next agent inherits a cleaner version of the story than the first agent actually knew. After a few handoffs, a tentative note can become a decision nobody made. Better models did not solve that for us; preserving the difference between what had been authorized and what had merely been guessed did.

None of this makes the runtime the authority. A record cannot decide whether an action was legitimate; it can only make the question answerable. That distinction, between the infrastructure that proves what happened and the judgment about whether it should have, is the whole game, and it is why the inspectable runtime is a precondition for letting agents do more, not a substitute for deciding how much they should do.

The objection

The first objection is that inspection can become performance. Add a verifier, and you have added another system that has to be trusted. If that verifier is shallow, gameable, or wrong in ways nobody notices, the audit trail may look more convincing without making the system more accountable. The second is friction. Demand a full evidentiary record for every action an agent takes and you can drown a useful tool in paperwork, turning a system that was supposed to save work into one that generates it.

The answer is not to inspect everything at maximum resolution. It is to match the record to the consequence of the action. Reliability engineering already learned a version of this: systems do not keep every trace forever at full fidelity. They spend attention where failures are most likely to matter. An agent answering a question about store hours needs almost no record at all. An agent moving money or editing a medical chart needs the whole thing. The more damage a wrong action can cause, the stronger the record needs to be.

Back to Monday morning

The engineer reading the Monday morning message will get a better dashboard eventually. It may show more steps, more timings, more tool calls, more traces. But it still will not answer the compliance question unless the runtime recorded the agent's permission at the moment it acted.

The system has to preserve what mattered while the action was still happening: what the agent saw, what it was allowed to do, and who or what gave it that authority. Otherwise the most important question arrives after the evidence is already gone.