Skip to content
Eval Suites50

GPQA Diamond

Graduate-level biology, physics, and chemistry questions written to resist web search. Diamond is the hardest subset.

GPQA Diamond at a glance

Maintainer
David Rein and GPQA authors
Size
198 questions[1]
Category
Reasoning & Frontier
Runs with
Inspect AI
License
MIT
Tier
production

Failure modes GPQA Diamond catches

Expert-level errors
Guessing

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

GPQA Diamond starter
pip install inspect-ai inspect-evals
inspect eval inspect_evals/gpqa_diamond --model openai/gpt-4.1 --limit 20

Sources

  1. 1. GPQA Diamond official page or repository · David Rein and GPQA authors · Repository · Sources checked
  2. 2. GPQA Diamond paper · arXiv · Paper · Sources checked