SWE-Lancer
Real Upwork freelance software tasks valued at $1 million total, including manager-style proposal selection, graded by end-to-end tests.
Failure modes SWE-Lancer catches
Incomplete features
UI regressions
Bad tradeoff choices
Starter command
A starting point, not a guarantee. Commands change between releases, so confirm against the official README.
git clone https://github.com/openai/SWELancer-Benchmark
# follow the repo README for setup and run commandsSources
- 1. SWE-Lancer official page or repository · OpenAI · Repository · Sources checked
- 2. SWE-Lancer paper · arXiv · Paper · Sources checked