Skip to content
Eval Suites50

Humanity's Last Exam

Expert-written questions across dozens of subjects, built to sit at the edge of current model ability.

Humanity's Last Exam at a glance

Maintainer
Center for AI Safety; Scale AI
Size
2,500 questions[1]
Category
Reasoning & Frontier
Runs with
Inspect AI
License
Not stated by the source. Check before commercial use.
Tier
frontier

Failure modes Humanity's Last Exam catches

Knowledge gaps
Overconfidence

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

Humanity's Last Exam starter
pip install inspect-ai inspect-evals
inspect eval inspect_evals/hle --model openai/gpt-4.1 --limit 20

Sources

  1. 1. Humanity's Last Exam official page or repository · Center for AI Safety; Scale AI · Repository · Sources checked
  2. 2. Humanity's Last Exam paper · arXiv · Paper · Sources checked