Humanity's Last Exam
Expert-written questions across dozens of subjects, built to sit at the edge of current model ability.
Failure modes Humanity's Last Exam catches
Knowledge gaps
Overconfidence
Starter command
A starting point, not a guarantee. Commands change between releases, so confirm against the official README.
pip install inspect-ai inspect-evals
inspect eval inspect_evals/hle --model openai/gpt-4.1 --limit 20Sources
- 1. Humanity's Last Exam official page or repository · Center for AI Safety; Scale AI · Repository · Sources checked
- 2. Humanity's Last Exam paper · arXiv · Paper · Sources checked