An agent workflow can produce a correct final answer after taking an unacceptable route. It might call an expensive tool repeatedly, access the wrong account, or send the same notification twice. A final-answer score alone will miss those failures.
The evaluation harness therefore needs three views: did the task succeed, did the system follow its operating constraints, and what happened outside the conversation? Splitting the workflow across more agents does not remove those responsibilities. It creates more boundaries where they need checking.
Use a task fixture, not just a prompt
A useful fixture includes the starting state, the user's request, available tools, permission context, expected outcomes, and forbidden effects. For a hypothetical meeting assistant, the fixture might include a calendar snapshot and a rule that invitations require approval.
The expected result need not prescribe one exact sequence of steps. Two valid workflows may reach the same outcome. Test the invariants that matter: no overlapping booking, no unauthorized recipient, and no external send before approval.
Capture structured events
Record tool names, validated arguments, result summaries, timing, and state transitions under a shared run identifier. Preserve enough evidence to explain a failure without collecting every available secret or personal field. Logging needs a retention and access policy.
Use trace checks for empty retrieval, repeated delegation, invalid arguments, and exhausted budgets. Keep these diagnostic metrics alongside the end-to-end result. Passing each local step does not prove that the complete workflow met the user's goal.
Separate deterministic checks from judgments
Code can check schemas, allowed tool names, authorization outcomes, and whether a write happened more than once. Humans or carefully calibrated model judges may be needed to assess whether a summary distorted a source or whether the final answer resolved an ambiguous request.
Version the rubric as well as the model and prompt. Otherwise a score change can come from a changed judge rather than a changed application. Preserve disagreement examples so the team can improve the rubric rather than repeatedly arguing about one opaque number.
Replay carefully
Recorded tool responses make regression tests cheaper and more reproducible. However, a cache keyed only by model, prompt, and sampling parameters is incomplete for a stateful workflow. The effective input can also include conversation history, retrieved context, tool definitions, authorization scope, and environment state.
A replay tests behavior against a recorded world; it does not prove that today's external integration works. Keep a smaller integration suite for live contract checks, with isolated test accounts and controlled side effects. Never replay a production write simply to reproduce a score.
Make failure and recovery part of the fixture
- Return a timeout after a tool may already have completed its operation.
- Make one dependency unavailable and check the fallback.
- Supply conflicting records and verify that the conflict survives the summary.
- Limit the number of steps and the total execution budget.
- Ask for an action outside the user's permissions.
Run stochastic cases more than once and report variation instead of selecting the best trace. A release report should contain outcome rates, constraint violations, cost and latency distributions, and representative changed cases. If the report cannot explain why a candidate is safer or more useful than the current workflow, the harness is not yet doing its job.
Related: inspectable agent interfaces and the prototype-to-production boundary.
Keep repeated success separate from best-of-many
A task that succeeds at least once in several attempts has a different operating profile from a task that succeeds on every attempt. Label best-of-many capability separately from repeated-run consistency. Reset the starting state between independent trials, include the cost of failed attempts, and identify which effects an outcome validator checks. A passing natural-language answer is not evidence that the intended record was changed correctly.