How Fast Can an Agent-Run Company Grow?

This article uses a fictional agent-run software business to introduce research conducted by Future Shock into how reliably models can check one another and what their failures might mean for a growing company.

Sarah runs a company that connects small retailers’ online stores to their accounting software. When a shop sells a jacket, collects sales tax, or refunds a purchase, her software records each transaction in the right ledger account. Millions of shops sell through platforms like Shopify, and several companies already offer connections to accounting systems such as QuickBooks and Xero. Sarah’s service configures those connections for each customer’s accounts, tax setup, and refund rules. Her engineers used to spend a couple of weeks writing a connector for each new customer. Her agents now read the two systems’ interfaces, build the mapping between them, and test it against last quarter’s transactions before putting it into use. A shop that signs up in the morning can usually have its transactions syncing by the afternoon. The agents also maintain her app-store listing, answer prospective customers’ questions, guide them through trials, and follow up with those who haven’t finished signing up. Sarah added forty stores last month without hiring anyone.

Customers pay a subscription to keep those connections running as their systems change. A shop owner who used to spend Sunday evenings transferring transactions into the accounts can now open the accounting software and find them already recorded. When the store platform changes how it reports a partial refund, the agents detect the failed sync, read the new format, adjust the mapping, and reprocess the affected transactions before anyone at the shop has to ask. They maintain the existing customers’ connections while setting up new ones, so adding stores does not leave Sarah with a growing maintenance backlog.

Each new subscription includes a promise to cover certain costs if the software makes a mistake. If a connector puts transactions in the wrong accounts, Sarah’s company will correct the books and pay for the accountant’s time to reconcile them. That assurance helps persuade a shop owner to sign up, because they know who will pay if the connection goes wrong. Sarah can offer the same terms to another forty stores while her agents set up their connections. Whether she can afford those commitments depends on what the corrections could cost and how much money she has available to cover them.

Who backs the connectors

Sarah can keep a reserve and pay for the occasional reconciliation out of revenue, which works while the errors stay small and rare. If she wants to keep offering the same promise as the company grows, she may need more money behind it than she can set aside herself. She could bring in more capital or pay another company to share the risk through a guarantee or insurance. Munich Re already sells a product that backs performance warranties on AI systems, and its own description of the problem is close to Sarah’s. “Providing a performance warranty for the AI models accuracy de-risks the purchase decision,” the product page says, “but it creates a liability on the balance sheet of the tech company.” Whether her particular contract would qualify is a separate question. Sarah would be paying for the ability to offer customers a promise she could not afford to make on her own.

To underwrite Sarah’s promise, a backer has to estimate how often her software might make a mistake, what the mistake could cost, and how much of that cost the contract would cover. For her connectors, the bill also depends on when the error is caught. Suppose a migration maps the sales-tax field into a revenue account. If the agents catch that error while testing the connector against last quarter’s transactions, they can fix the mapping before any live books are affected. If it surfaces three weeks later, after the shop's accountant has prepared a quarterly return from books that overstate revenue and understate tax, the agents can still find and reverse every affected entry overnight. The accountant still has to revise the return prepared from those incorrect entries, and someone has to pay for that time. Sarah has agreed to cover those reconciliation costs, and an underwriter has to price the accountant’s bill as well as the software repair.

Seen from the backer's side, forty separate stores look like a good thing. A reserve or policy covering many small, unrelated exposures is easier to size than one covering a single large exposure. The errors don’t all arrive at once, so a growing customer list can make Sarah’s commitments easier to back. The expected losses become predictable relative to the whole book, and predictable losses can be priced. If Sarah's forty stores fail independently of each other, the backer has a fair chance of estimating the total bill from how often mistakes happen and what each one costs.

But Sarah’s stores may not fail independently. Suppose the bad mapping came from a shared migration example, a published pattern for connecting this kind of store to this kind of ledger, and her agents adopted it across several connectors. The tests they wrote came from the same example, so the expected values inherited the wrong mapping and the tests passed. The error could then sit in several customers’ books, passing its checks, until an accountant noticed. Other companies using the same example could carry the same fault. A firm that checked its mappings against accounting expectations established independently of the example might catch it. Tests derived from the faulty example alone would not expose the mistake. This does not require every connector to fail, or all of them to fail on the same night. Separate customers, even at separate companies, could incur correction costs because their software inherited the same error.

Insurers already have to account for dependencies shared by otherwise separate businesses. In 2018 Lloyd’s commissioned a study of what a major cloud outage would cost. Rather than estimating exposure from cloud providers’ market share, the modelers used industry exposure records to trace which insured businesses depended on which vendors, an approach the report calls “detailed accumulation.” A backer could face claims from several otherwise separate companies if they all depended on the same cloud provider and it failed. For Sarah’s backer, counting customers would not be enough. It would need to know how many connectors inherited the faulty mapping and whether other companies it covered had used the same example. Coverage might still be possible, but the backer would have to account for several companies incurring costs from the same mistake. How could the backer tell whether it was covering separate risks or repeated exposure to the same mistake?

When models fail together

Our research began with the suspicion that companies relying on the same foundation models could make similar mistakes. If Sarah and her competitors all used the same handful of models, a backer covering four firms might be taking on more shared risk than it realized. Researchers were already studying correlated errors across language models. We wanted to understand how much independence a company could expect from using different models, and whether that justified treating their failures as separate risks.

Our first study weakened that worry considerably. Across a large retrospective panel of models answering the same questions, raw agreement on failures was high, but most of it was consistent with the questions simply being hard for many models at once. After adjusting for item difficulty, the typical residual overlap between models sat near zero, and a fresh prospective panel kept only a small positive remainder. Two maps describe a system like Sarah's. One shows which models it contains; the other shows which inputs those models share. In the tested contrasts, swapping the model endpoint decorrelated failures more than swapping the evidence source did, and that substitution changed a whole endpoint bundle rather than vendor identity alone. The evidence did not show that shared inputs mattered more than model choice, but knowing which models a system used was still not enough to tell us how often they would fail together.

We then asked whether models would be more likely to fail together if they received the same faulty input. Sarah’s migration example helps explain why we wanted to know, but the experiment tested responses to errors in synthetic records. A faulty coding pattern can be copied into several connectors, while a cloud outage can interrupt several businesses at once. Neither is the same as models failing to correct an erroneous field in a test record. The experiment let us investigate how a shared input could link failures, but applying its findings to those business situations would require evidence about how each failure occurs and spreads.

The follow-up study fixed eight model and deployment routes and sent each of them synthetic records with one recoverable wrong field. In one condition every route received the identical corruption. In the other, each route drew its corruption independently from the same pool, so two routes could still land on the same wrong variant by chance. The routes had to flag the error, name what was wrong, supply the correct value, and emit a specific repair object; a route could return the right final answer and still fail that repair contract. These were fixed endpoints answering arithmetic, constraint, extraction, and tool-call items, with code tasks excluded before collection after a calibration failure. The result applies to that panel and nothing wider.

Across the same 1,677 item families, the average share of routes failing exact repair barely moved between the two conditions, moving from 59.2 to 59.9 percent, with an interval that includes no change at all. The number of item families where all eight routes failed the repair rose from 77 to 117 of that 1,677. The average barely moved while the all-eight extreme moved more. That is the shape a backer worries about, a portfolio where the losses arrive together, though these are failures on a benchmark's repair contract, not damaged books.

The findings are preliminary, but they show why an average failure rate cannot tell us how often every route will fail on the same case. A system that relies on one route catching another’s mistakes needs to measure those shared failures directly.[1]

The company behind the agents

For improvements in models and harnesses to produce exponential business growth, they would have to do more than let a company produce software faster. The company would also need to find customers, deliver what it promised, and take responsibility for an expanding volume of decisions made on its behalf. Some of those demands could become easier to meet as the agents improve. Better agents might test more thoroughly, catch mistakes sooner, and restore systems faster, while new revenue helps fund reserves and insurance.

As agents take on more operations and decisions, the company needs to know whether its checks and recovery procedures can keep up. For Sarah, that means testing new mappings before they reach live books and catching errors before customers use the records to prepare returns. A test developed independently of the example that produced the code might catch a mistake that several agents repeat. But a checking process that handles forty connectors may need to change to handle thousands, especially if one faulty update can reach many customers at once. Better models and harnesses could help automate those checks and repairs. Whether they improve quickly enough to keep pace with production is something the company would have to measure in operation; the repair study does not answer it.

Financial backing could expand alongside those improvements. Growing revenue can fund larger reserves, while an operating record gives insurers and other backers evidence about how often mistakes occur, how far they spread, and what they cost to correct. The Geneva Association’s report on insuring generative AI risk describes insurers considering maintenance and monitoring requirements, including how to “embed continuous monitoring in policy terms,” while noting the difficulty of verifying how businesses manage these risks. If a company could demonstrate that its controls remained effective as its agents handled more decisions, that evidence might help it obtain backing for further growth. Independent checks and rehearsed recovery could then support both safer operations and better coverage terms, although the report does not establish that insurers offer discounts for those particular practices.

A company could also grow by narrowing what it promises to cover. Sarah might cap reconciliation payments, exclude some consequential costs, or require customers to check the records themselves. Where those terms are enforceable, they could reduce her company’s exposure while leaving shop owners with more of the checking and repair costs. That would make growth easier for her business without necessarily making the overall process more productive. The distinction matters when moving from one company to the economy. An agent-run business can grow by winning customers from competitors or shifting costs outside its accounts. To connect exponential improvements in models and harnesses to economic growth, we would need to see more goods and services produced across the economy, accounting for the resources spent checking decisions and putting mistakes right—not just faster growth in the companies deploying agents.

[1] Study limitations. These comparisons were postconfirmatory analyses in a working paper that has not been peer reviewed. The study did not meet its full confirmation criteria: the implemented analysis used different weights from the protocol, and a required precollection check on statistical support was omitted and fails when reconstructed afterward. The unadjusted percentile intervals for the differences were −0.12 to 1.45 percentage points for average exact-repair failure and 0.95 to 3.82 for all-eight failure. The primary dependence contrasts for error detection and final-answer failure both had intervals that included no change. Not every collective-failure threshold worsened; the share of cases where at least half the routes failed decreased slightly. The plots use different vertical scales, and these benchmark outcomes do not establish real-world incident rates or costs. Full methods and results.