tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

Catalog evidence and recorded chart appearances. Inclusion is not an endorsement.

Source snapshot imported 2026-10-20. This is not a live feed. Source and contributions.

Evidence and limits

Finding: Simulated users, domain policies and final database state test outcome correctness and repeated-run consistency.

Impact: Watch the recorded review topic (see source), the recorded review topic (see source), the recorded review topic (see source).

Boundary: Model-simulated users are not human participants; goal-state matching does not capture every valid interaction. Current tau2 repository contains newer grading and domains, not this paper's exact evaluated release. Next: Pin task and grader revisions and compare repeated-run outcomes without mixing incompatible benchmark versions. Added candidate with Selected full-text review; not an exhaustive evaluation appraisal; not full-text validated, replicated or adopted

Flags: Selected full-text review; not an exhaustive evaluation: Model-simulated users are not human participants; goal-state matching does not capture every valid interaction. Current tau2 repository contains newer grading and domains, not this paper's exact evaluated release.

Record State: Candidate only: not accepted into the reviewed catalog

What was reviewed: Selected full-text review; not an exhaustive evaluation

Inspect original source

Recorded chart appearances

  • 2026-10-13: position 20. Relevant to the recorded review topic (see source); Selected full-text review; not an exhaustive evaluation evidence
  • 2026-10-12: position 19. Relevant to the recorded review topic (see source); Selected full-text review; not an exhaustive evaluation evidence