Skip to content
Eval Suites50

Cybench

Professional capture-the-flag challenges from four competitions, with optional subtasks for partial credit.

Cybench at a glance

Maintainer
Cybench authors / Stanford
Size
40 CTF tasks[1]
Category
Security & Red-Teaming
Runs with
Inspect AI
License
Apache-2.0
Tier
frontier

Failure modes Cybench catches

Exploit failures
Recon gaps
Tool misuse

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

Cybench starter
pip install inspect-ai inspect-evals
inspect eval inspect_evals/cybench --model openai/gpt-4.1 --limit 20

Sources

  1. 1. Cybench official page or repository · Cybench authors / Stanford · Repository · Sources checked
  2. 2. Cybench paper · arXiv · Paper · Sources checked