What it means for API owners

One sandbox, two tests, one run per configuration. Findings at that scale are observations, not laws. These are the four we would act on if this API were ours, each tied to the moment in the runs that produced it.

1. Machine-readable docs changed the cost, not the outcome

Both builder runs shipped the same working app. The difference was spend: half the messages, half the tool calls, a third of the errors, first successful API call in about two minutes. The surfaces behind that difference (raw markdown per page, /llms.txt, a typed /openapi.json, three small doc tools) were a few days of work on top of content that already existed. Same corpus, different access. For an API expecting agent traffic, that effort-to-savings ratio is the thing to evaluate against your own numbers.

2. Agents ask documentation questions; structure decides whether answers land

The human-style agent downloaded every page and reconstructed the contract before writing code. The agent-native one asked: one listing call, then seven targeted reads. What made the targeted reads land was mostly the corpus behind the tools: one page per concern, every outcome reachable from the index, examples anchored to a single seeded scenario. Retrieval was only as good as the structure underneath it.

3. Error messages are part of the interface

The cleanest self-correction in either test came from a 422 that named its violation: “all offer_selections must be sourced from a single supplier”. The agent re-read the basket, regrouped its order, and placed again, with no help. An error that states what was violated and what to do next worked, in practice, as documentation delivered at the exact moment it was needed.

4. Agents find the gaps humans route around

The install call returned credentials but no webhook secret. A human integrator would email support and move on; both agents hit the wall, proved it, and surfaced it as a contract gap, along with two genuine bugs in the sandbox. An agent driving a contract end to end exercises paths a happy-path demo never touches. Two agent runs produced more contract verification than all of our own prior testing.

The scope, again

All of this comes from a compact API, a deterministic seeded scenario, and one run per configuration. The mechanisms looked durable; the magnitudes are not portable. The way to trust any of it is to run the tests yourself or read the raw evidence.