Skip to content
Eval Suites50

How to test AI agents: mock tools, bound loops and verify state

Sources checked 3 min read · By the Eval Suites editors

In brief

Test an AI agent by fixing its starting state, controlling its tools and checking the resulting state and trace. Validate tool arguments in code and enforce execution budgets outside the model. τ-bench demonstrates final-state scoring; Inspect AI documents agent execution limits.[1][2]

Test state
Reset per case; fictional API records
Loop rule
Explicit application-specific budgets
Success signal
Observed permitted effect, not narrated success

How do you test an autonomous agent that calls external APIs?

Replace external APIs with deterministic fixture tools for regression tests, then separately test real integrations in an authorized sandbox. τ-bench evaluates tool interactions against goal state; Inspect AI supports sandboxed environments when tasks need execution isolation.[1][3]

  1. Define the initial records and permissions for each case; reset them before the next run.
  2. Replace payment, email and destructive writes with fixtures that record attempted calls.
  3. Return controlled success, timeout, conflict and malformed-response outcomes.
  4. Run the agent and verify allowed effects, forbidden effects and any final answer claims against actual fixture state.
  5. Retain a separate integration suite for the real service contract. Passing a mock does not prove credentials, quotas or a hosted service work.
Executable fictional API fixture; connect it to your own agent tools
class FixtureOrders:
    def __init__(self):
        self.refunded = False
        self.calls = []

    def refund(self, order_id):
        self.calls.append({"name": "refund", "order_id": order_id})
        if order_id != "fixture-1042":
            raise ValueError("unknown fixture order")
        if self.refunded:
            return {"status": "already_refunded"}
        self.refunded = True
        return {"status": "refunded"}

orders = FixtureOrders()
assert orders.refund("fixture-1042")["status"] == "refunded"
assert orders.refund("fixture-1042")["status"] == "already_refunded"
assert orders.refunded is True

The fixture is a proposed test design, not a payment integration. Add separate assertions for whether duplicate calls were attempted and whether duplicate effects occurred; those are different failure signals.

How do you detect infinite tool execution loops in agent evals?

Enforce a hard execution budget and track repeated calls without progress. Inspect AI documents token, message and time limits; a repetition detector is an additional application rule, not a replacement for those limits.[2]

Normalize tool name and arguments before comparing repeats. Include state changes: polling the same job while its status advances is different from asking the same failed query forever. The example below chooses three identical calls without a state-version change as its own test rule, not a recommended universal threshold.

Proposed no-progress rule; enforce a separate global step and time budget
import json

def repeated_without_progress(calls, limit=3):
    streak, previous = 0, None
    for call in calls:
        key = (call["name"], json.dumps(call["arguments"], sort_keys=True),
               call["state_version"])
        streak = streak + 1 if key == previous else 1
        if streak >= limit:
            return True
        previous = key
    return False

call = {"name": "lookup", "arguments": {"id": "fixture-1042"}, "state_version": 0}
assert repeated_without_progress([call, call]) is False
assert repeated_without_progress([call, call, call]) is True

A nonconsecutive cycle can evade this streak check. Add a global maximum step count, deadline and spend budget in the runner, and test alternating calls and changing arguments as well. Keep cancellation and failure results explicit so a timed-out run is never counted as success.

What deterministic assertions should you use to test JSON tool call arguments?

Check schema shape, field values, authorization and the resulting side effect separately. JSON Schema supports required properties and rejecting extra properties; a valid object still does not prove that the caller can perform the requested action.[4]

Illustrative refund-argument schema; fixture amount is not a site policy
{
  "type": "object",
  "required": ["order_id", "amount_cents"],
  "additionalProperties": false,
  "properties": {
    "order_id": {"type": "string", "enum": ["fixture-1042"]},
    "amount_cents": {"type": "integer", "minimum": 1, "maximum": 3900}
  }
}

Validate the actual tool argument object before execution, then check permissions against trusted state. Reject an unexpected tool name, extra fields or an amount beyond the fixture’s allowance. Promptfoo documents JSON and JavaScript assertions for programmable checks; adapt them to the output and trace contract you actually expose.[5][4]

Which stateful failure cases belong in the first agent test suite?

Include interrupted and repeated actions, not only the first successful request. This is a proposed application test matrix; choose pass conditions from your real business rules.

Proposed agent regression matrix
CaseControlled inputExpected evidence
Retry after timeoutUnknown completion status from a fixture APIReconcile state before another write
Duplicate requestSame operation ID submitted againNo duplicate side effect
Permission failureUser outside the permitted accountNo unauthorized tool effect
Malformed argumentsMissing or unexpected JSON fieldsValidation fails before execution
No-progress cycleRepeated unchanged tool resultRunner stops with an explicit failure
False success claimAgent narrates completion without a writeEnd-state assertion fails

Use the existing CI-gating guide to decide when this suite runs. Keep judge-based style checks separate from deterministic permission, schema and state checks so a fluent answer cannot conceal a failed operation.

Related suites

Related reading

Sources

  1. 1. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains · arXiv:2406.12045 (Sierra) · Paper · Sources checked
  2. 2. Agents and execution limits · UK AI Security Institute · Official docs · Sources checked
  3. 3. Inspect AI: Sandboxing · UK AI Security Institute · Official docs · Sources checked
  4. 4. JSON Schema object validation · JSON Schema · Official docs · Sources checked
  5. 5. Assertions and metrics · Promptfoo · Official docs · Sources checked