Skip to content
Eval Suites50

TruthfulQA

Questions where common human misconceptions are the tempting answer; measures whether models repeat falsehoods.

TruthfulQA at a glance

Maintainer
Stephanie Lin; Jacob Hilton; Owain Evans
Size
817 questions[1]
Category
RAG & Factuality
Runs with
Inspect AI
License
Apache-2.0
Tier
starter

Failure modes TruthfulQA catches

Imitative falsehoods
False premises

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

TruthfulQA starter
pip install inspect-ai inspect-evals
inspect eval inspect_evals/truthfulqa --model openai/gpt-4.1 --limit 20

Sources

  1. 1. TruthfulQA official page or repository · Stephanie Lin; Jacob Hilton; Owain Evans · Repository · Sources checked
  2. 2. TruthfulQA paper · arXiv · Paper · Sources checked