Skip to content
Eval Suites50

Reasoning & Frontier eval suites

Reasoning and frontier suites use expert-written or unpublished problems to measure capability at the edge, with extra care about training-data contamination.

6 of 6 suites

FrontierMath

frontier

Original, unpublished research-level math problems with automatically checkable answers. Most of the set is private.

Size
350 problems
Runs with
Gated (Epoch AI)
License
Not stated
Category
Reasoning

Expert-written questions across dozens of subjects, built to sit at the edge of current model ability.

Size
2,500 questions
Runs with
Inspect AI
License
Not stated
Category
Reasoning

ARC-AGI-2

frontier

Abstract grid puzzles that require inferring new rules from a few examples, aimed at fluid reasoning over memorization.

Size
1,000 training · 120 public eval
Runs with
Python
License
Not stated
Category
Reasoning

GPQA Diamond

production

Graduate-level biology, physics, and chemistry questions written to resist web search. Diamond is the hardest subset.

Size
198 questions
Runs with
Inspect AI
License
MIT
Category
Reasoning

AIME 2025

starter

The 30 problems from the 2025 AIME I and II exams, integer answers from 0 to 999.

Size
30 problems
Runs with
Inspect AI
License
Not stated
Category
Reasoning

MMLU-Pro

starter

Harder MMLU with 10 answer options and more reasoning-heavy questions across 14 subjects.

Size
12,032 questions
Runs with
Inspect AI
License
Not stated
Category
Reasoning

Size and license come from each suite's primary source. Not stated means the source doesn't declare a license; check the repo before commercial use.