Cybench
Professional capture-the-flag challenges from four competitions, with optional subtasks for partial credit.
Failure modes Cybench catches
Exploit failures
Recon gaps
Tool misuse
Starter command
A starting point, not a guarantee. Commands change between releases, so confirm against the official README.
pip install inspect-ai inspect-evals
inspect eval inspect_evals/cybench --model openai/gpt-4.1 --limit 20Sources
- 1. Cybench official page or repository · Cybench authors / Stanford · Repository · Sources checked
- 2. Cybench paper · arXiv · Paper · Sources checked