HealthBench
productionHealth conversations scored against physician-written rubrics for accuracy, safety, communication, and context-seeking.
- Size
- 5,000 conversations
- Runs with
- Inspect AI
- License
- MIT
- Category
- Domain
Domain suites test finance, health, legal, and accessibility tasks where wrong answers carry real cost, often with expert-written rubrics.
6 of 6 suites
Health conversations scored against physician-written rubrics for accuracy, safety, communication, and context-seeking.
Financial-analyst tasks using SEC filings. The harness is open source; only 50 questions are public, the rest are private or licensable.
Collaboratively built legal reasoning tasks: issue spotting, rule recall, interpretation, and rhetorical analysis. Licenses vary by task.
Multiple-choice questions from medical licensing exams, testing clinical knowledge and diagnostic reasoning.
Numerical reasoning over text and tables from earnings reports, with annotated reasoning programs.
Runs automated accessibility checks on HTML that LLMs generate. Automated checks catch many issues but don't prove full WCAG conformance.
Size and license come from each suite's primary source. Not stated means the source doesn't declare a license; check the repo before commercial use.