Skip to content
Eval Suites50

Terminal-Bench 2.1

Hard tasks in real terminal environments, from compiling code to configuring servers, run through the Harbor framework.

Terminal-Bench 2.1 at a glance

Maintainer
Terminal-Bench team; Stanford and Laude Institute
Size
89 tasks[1]
Category
Agents & Tool Use
Runs with
Harbor
License
Apache-2.0
Tier
frontier

Failure modes Terminal-Bench 2.1 catches

Shell command errors
Environment misconfig
Giving up early

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

Terminal-Bench 2.1 starter
pip install harbor
# see github.com/harbor-framework/terminal-bench-2-1 for the dataset id
harbor run --help

Sources

  1. 1. Terminal-Bench 2.1 official page or repository · Terminal-Bench team; Stanford and Laude Institute · Repository · Sources checked
  2. 2. Terminal-Bench 2.1 paper · arXiv · Paper · Sources checked