Skip to content
Eval Suites50

LiveCodeBench

Competitive programming problems collected over time with release dates, so you can score only problems published after a model's cutoff.

LiveCodeBench at a glance

Maintainer
LiveCodeBench authors
Size
Grows by release[1]
Category
Coding & Systems
Runs with
Python
License
MIT
Tier
production

Failure modes LiveCodeBench catches

Contamination
Edge-case failures
Timeouts

Starter command

A starting point, not a guarantee. Commands change between releases, so confirm against the official README.

LiveCodeBench starter
git clone https://github.com/LiveCodeBench/LiveCodeBench
# follow the repo README for setup and run commands

Sources

  1. 1. LiveCodeBench official page or repository · LiveCodeBench authors · Repository · Sources checked
  2. 2. LiveCodeBench paper · arXiv · Paper · Sources checked