AbstentionBench
Collects 20 datasets of unanswerable, underspecified, false-premise, and stale questions to test whether models abstain. Noncommercial license.
Failure modes AbstentionBench catches
Answering the unanswerable
Over-refusal
Starter command
A starting point, not a guarantee. Commands change between releases, so confirm against the official README.
git clone https://github.com/facebookresearch/AbstentionBench
# follow the repo README for setup and run commandsSources
- 1. AbstentionBench official page or repository · Meta Platforms, Inc. and affiliates · Repository · Sources checked
- 2. AbstentionBench paper · arXiv · Paper · Sources checked