Skip to content
Eval Suites50

FRAMES

Multi-hop questions that combine several Wikipedia articles, testing retrieval, reasoning, and factuality together.

FRAMES at a glance

Maintainer
Google
Size
824 questions[1]
Category
RAG & Factuality
Runs with
Python
License
Apache-2.0
Tier
production

Failure modes FRAMES catches

Multi-hop gaps
Temporal errors
Numeric reasoning

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

FRAMES starter
pip install datasets
python -c "from datasets import load_dataset; print(load_dataset('google/frames-benchmark'))"

Sources

  1. 1. FRAMES official page or repository · Google · Repository · Sources checked
  2. 2. FRAMES paper · arXiv · Paper · Sources checked