GAIA
General-assistant questions that need web browsing, file reading, and multi-step tool use, with short exact answers.
Failure modes GAIA catches
Shallow search
Tool chaining errors
Answer format drift
Starter command
A starting point, not a guarantee. Commands change between releases, so confirm against the official README.
pip install inspect-ai inspect-evals
inspect eval inspect_evals/gaia --model openai/gpt-4.1 --limit 20Sources
- 1. GAIA official page or repository · Meta; Hugging Face; AutoGPT · Repository · Sources checked
- 2. GAIA paper · arXiv · Paper · Sources checked