Coaching the Agent

An athlete practicing with a coach may already know what they want to improve. They’re trying to stay balanced, hold their form, or stop rushing a movement. But concentrating on what they intend to do can make it hard to notice what they actually do. The coach watches and points out a detail they missed.

Sometimes it's a problem, a habit that creeps in when they get tired. Sometimes the timing came together, but the athlete didn't notice what they did differently and can't yet repeat it on purpose. Either way, the coach gives them something specific to pay attention to on the next attempt, and they go back out to see what happens.

When an AI agent returns a result, it's easy to judge it without examining how it got there. We decide whether it's good enough, and if it isn't, we ask for another version, explain more carefully what we wanted, or give up and do it ourselves. The choices that shaped the result happened earlier, as the agent selected information, revised its answer and checked its claims. You can't see the model's private reasoning, but you can examine its sources, the evidence it used to support its recommendation and the checks it ran. Those choices are the agent's technique, and you can examine them together.

One Session

Consider an illustrative case. Someone on a five-person team asks an agent for a short briefing comparing three project-management tools. A colleague will make the decision, so the briefing needs to be fair and quick to read.

What comes back is well organized, and it includes something the person hadn't thought to ask about. The tools differ less in their features than in whom they're built for. Some are designed around the person tracking work, with dashboards, status reports and views of who is falling behind. Others are designed around the people doing work, with less overhead for updating a task and fewer fields to fill in. For a team of five, that distinction might matter more than any feature list.

The briefing strongly favors the best-known tool, one designed for the person tracking progress. Most of the material the agent found concerns that product, with far more reviews, comparisons and write-ups than it found for the other two. The briefing compares three tools, but the research behind it mostly covered one.

You could accept the recommendation, since it's well argued, or rewrite it yourself. Taking the coach's approach, you could instead identify what was useful in the briefing and ask why the recommendation didn't follow it. "The split between tools for tracking work and tools for doing it is the most useful part of this. For five people doing their own work, it points away from the tool you recommended. What pulled you back toward it?"

The agent says there was far more written about the popular tool. Since an agent's explanation of its choices may not match what it actually did, the person checks the sources it used. In this case, they support the explanation. The amount written about the popular tool had counted for more than the agent's distinction between whom the tools were designed for.

The person's first instinct is to ask for equal coverage of all three tools. That would make the briefing look balanced, but they haven't yet established whether its preference for the popular tool is wrong. For the next attempt, they ask the agent to use its distinction between tracking and doing to assess the recommendation. Does the popular tool fit this team, or does the available evidence just make it easier to recommend? They leave the agent to decide how to investigate.

Rather than hunting for more reviews of the two less popular tools, the agent uses its distinction to examine the reviewers. It sorts the reviews it already has by the writer's role and team size, an approach the person hadn't considered. In this set of reviews, most of the praise for the popular tool comes from people at large organizations, many of them managers. Their reviews describe how the tool helps them track other people's progress, which says relatively little about how it would serve five people managing their own tasks.

The agent changes its recommendation to one of the tools designed for people managing their own tasks. By the criterion it set out in the briefing, that tool fits the team better, though the evidence for it amounts to only a handful of reviews from small teams. It suggests trying the tool on a real project for a couple of weeks before deciding. Asking for equal coverage would have addressed how much the briefing said about each product without addressing whose experience the reviews described.

In MINT, a benchmark that tests models working through multi-step problems with tools, models generally improved more when given written feedback during a task than when told only whether they were right. GPT-4 supplied the feedback, playing the part of a user, and the models receiving it are now a few generations old. The study didn't compare asking questions with giving instructions, so it offers evidence about the usefulness of feedback without establishing how a person should guide an agent.

The Wrong Tool

Coaching takes attention, and plenty of agent mistakes don't deserve it. If the briefing gets a price wrong or the formatting is off, fix it or say so and move on. Even when the problem goes beyond a simple correction, you may already know what you want changed. Saying so directly is faster than asking a question you know the answer to.

Some tasks need an exact instruction instead of a question. If an agent must never send anything to a client without approval, that rule belongs in its standing instructions or permissions from the first session. Establishing those rules and deciding what needs your review before it can be sent are part of setting up the agent, a subject we covered in Not One Prompt and Done. Coaching addresses choices that arise during a task and weren't anticipated in that setup.

The coach can also be wrong, and a responsive agent can follow bad guidance as well as good. In one MINT experiment, the researchers gave the model providing feedback a reference solution to compare against. Performance on simulated household tasks fell below the no-feedback baseline, even though the same change helped on reasoning and coding tasks. The authors suggest that when an agent took a different route, comparing its actions with the reference solution confused the model providing feedback. A coach can make a similar mistake by treating unfamiliar technique as bad technique.

Models also differ in how well they use guidance, and training can improve that ability. In one recent study, researchers trained models to make better use of feedback during a task. The improvements carried over to some unrelated tasks but not others. If you've repeatedly asked an agent to examine its approach and nothing changes, the difficulty may lie in its ability to use the feedback rather than in how you phrased the question.

The Next Briefing

The session produced a better briefing and a lesson that could help with other comparisons. Before trusting a recommendation, check where the evidence comes from as well as how much of it there is.

An agent agreeing to keep that lesson in mind doesn't mean it will have access to it in a new conversation. The lesson needs to be saved somewhere the agent will consult again, such as a project note, a standing instruction or a skill, a saved set of instructions it loads for a task. Otherwise, a fresh session starts without it.

The next briefing is where you find out whether saving the lesson helped. Say the team later needs a shared calendar tool. If the agent applies the lesson, it checks whose experience the reviews describe before recommending anything. When there's barely any evidence from teams their size, it says so up front rather than letting the best-reviewed option win by default.