The operator test: run the lifecycle

The builder test measured an agent writing code against the API. This test measures the other surface: an agent using the API to do the work itself, through the Procurement MCP’s tools, with no code and no HTTP.

The task

A fresh agent gets one connector, the Procurement MCP, and this prompt:

“Using only the Procurement MCP tools, take the repair job for the 2019 Toyota Corolla (external reference CCC-2026-04817) through to a reconciled invoice. Tell me what you did and why at each step.”

No documentation, no walkthrough. Whatever the agent learns about the platform has to come from the tool descriptions themselves.

What counts as a pass

With no human help, the agent must:

Complete the lifecycle: open the job, identify parts, get a supplier basket, place the order, have the supplier confirm it, reconcile the invoice.
Orient itself from the tool and resource descriptions alone.
Hit the planted error and recover from the error message (below).
End at a correctly reconciled invoice, with the steps that go beyond the public contract flagged as such.
Narrate clearly enough that an engineer reading the transcript understands the platform.

The planted error

The contract refuses an order that mixes suppliers. The test deliberately steers the agent into placing one. The API answers with a 422 whose message names the violation, “all offer_selections must be sourced from a single supplier”, and the recovery path. A passing agent reads the message, re-fetches the basket, and corrects its order without being told how.

What happened

The run took place on 5 June 2026.

The agent completed the full lifecycle unaided, ending at a reconciled invoice with every line matched.
It recovered from the planted 422 on its own: re-read the basket, regrouped its selections to a single supplier, placed again.
It flagged the two beyond-contract steps correctly each time it used them.

One caveat, stated plainly: the evaluating agent had earlier helped write the system’s build spec, so this run validates that the tools work and that errors are legible. It does not prove a stranger can do it. A later, uninvolved agent re-drove the full lifecycle through the builder test’s running app and reached the same deterministic invoice, with all four webhook deliveries verified.

The run also fed three fixes back into the sandbox, including the basket currency change described in Real vs. simulated. Details and the full play-by-play: The numbers and the run transcript.

Run it yourself

Add the Procurement MCP as a custom connector (claude.ai: Settings, Connectors, Add custom connector):
```
https://partifact-procurement-mcp.thanhvuttv.workers.dev/mcp/partifact-public-demo/<your-slot>
```
Replace <your-slot> with any name. It becomes your own private sandbox, seeded with the same Corolla job; pick a new name any time for a clean slate.
In a fresh chat, paste the prompt above.
A capable agent finishes unaided in one conversation. The invoice should come to $1,099.00 NZD if it orders the two Christchurch Toyota parts; a different total is worth reporting.

Connection options for Claude Desktop and Claude Code: Try it.

The numbers, where both tests are scored from the raw logs.