FrontierMath
frontierOriginal, unpublished research-level math problems with automatically checkable answers. Most of the set is private.
- Size
- 350 problems
- Runs with
- Gated (Epoch AI)
- License
- Not stated
- Category
- Reasoning
Reasoning and frontier suites use expert-written or unpublished problems to measure capability at the edge, with extra care about training-data contamination.
6 of 6 suites
Original, unpublished research-level math problems with automatically checkable answers. Most of the set is private.
Expert-written questions across dozens of subjects, built to sit at the edge of current model ability.
Abstract grid puzzles that require inferring new rules from a few examples, aimed at fluid reasoning over memorization.
Graduate-level biology, physics, and chemistry questions written to resist web search. Diamond is the hardest subset.
The 30 problems from the 2025 AIME I and II exams, integer answers from 0 to 999.
Harder MMLU with 10 answer options and more reasoning-heavy questions across 14 subjects.
Size and license come from each suite's primary source. Not stated means the source doesn't declare a license; check the repo before commercial use.