Skip to content
Eval Suites50

MMMU

College-level questions over charts, diagrams, maps, and scientific images across 30 subjects.

MMMU at a glance

Maintainer
MMMU team
Size
11,500 questions[1]
Category
Voice & Multimodal
Runs with
Inspect AI
License
Not stated by the source. Check before commercial use.
Tier
production

Failure modes MMMU catches

Visual misreads
Chart reasoning

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

MMMU starter
git clone https://github.com/MMMU-Benchmark/MMMU
# follow the repo README for setup and run commands

Sources

  1. 1. MMMU official page or repository · MMMU team · Repository · Sources checked
  2. 2. MMMU paper · arXiv · Paper · Sources checked