Skip to content
Eval Suites50

HealthBench

Health conversations scored against physician-written rubrics for accuracy, safety, communication, and context-seeking.

HealthBench at a glance

Maintainer
OpenAI
Size
5,000 conversations[1]
Category
Domain: Finance / Health / Legal
Runs with
Inspect AI
License
MIT
Tier
production

Failure modes HealthBench catches

Unsafe advice
Missing context
Poor escalation

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

HealthBench starter
pip install inspect-ai inspect-evals
inspect eval inspect_evals/healthbench --model openai/gpt-4.1 --limit 20

Sources

  1. 1. HealthBench official page or repository · OpenAI · Repository · Sources checked
  2. 2. HealthBench paper · arXiv · Paper · Sources checked