GPQA Diamond
Graduate-level biology, physics, and chemistry questions written to resist web search. Diamond is the hardest subset.
Failure modes GPQA Diamond catches
Expert-level errors
Guessing
Starter command
A starting point, not a guarantee. Commands change between releases, so confirm against the official README.
pip install inspect-ai inspect-evals
inspect eval inspect_evals/gpqa_diamond --model openai/gpt-4.1 --limit 20Sources
- 1. GPQA Diamond official page or repository · David Rein and GPQA authors · Repository · Sources checked
- 2. GPQA Diamond paper · arXiv · Paper · Sources checked