Skip to content
Eval Suites50

HumanEval+ (EvalPlus)

HumanEval's 164 problems with roughly 80x more tests, catching code that passed the original thin test suites.

HumanEval+ (EvalPlus) at a glance

Maintainer
EvalPlus team
Size
164 problems[1]
Category
Coding & Systems
Runs with
EvalPlus
License
Apache-2.0
Tier
starter

Failure modes HumanEval+ (EvalPlus) catches

Edge-case bugs
Off-by-one
Type errors

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

HumanEval+ (EvalPlus) starter
pip install evalplus
evalplus.evaluate --model gpt-4.1 --dataset humaneval --backend openai

Sources

  1. 1. HumanEval+ (EvalPlus) official page or repository · EvalPlus team · Repository · Sources checked
  2. 2. HumanEval+ (EvalPlus) paper · arXiv · Paper · Sources checked