Skip to content
Eval Suites50

DUMA-Bench

Extends τ²-bench with adversarial dual-control security scenarios, where the user or shared environment tries to manipulate the agent.

DUMA-Bench at a glance

Maintainer
Ivan Aleksandrov; German Kochnev; Sabrina Sadiekh; Yaroslav Rogoza; ITMO AI Security Lab / Hive Trace Lab
Size
8 security domains[1]
Category
Security & Red-Teaming
Runs with
Python
License
MIT
Tier
frontier

Failure modes DUMA-Bench catches

User-side manipulation
Policy bypass
Cross-agent attacks

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

DUMA-Bench starter
git clone https://github.com/123yaroslav/duma-benchmark
# follow the repo README for setup and run commands

Sources

  1. 1. DUMA-Bench official page or repository · Ivan Aleksandrov; German Kochnev; Sabrina Sadiekh; Yaroslav Rogoza; ITMO AI Security Lab / Hive Trace Lab · Repository · Sources checked
  2. 2. DUMA-Bench paper · arXiv · Paper · Sources checked