Task
Other
S
nebius/SWE-rebench
Other · nebius· 346.8K
cc-by-4.0
920 MB
Dataset Summary SWE-rebench is a large-scale dataset designed to support training and evaluation of LLM-based software engineering (SWE) agents, building upon and expanding our earlier release, SWE-bench-extra. It is constructed using a fully automated pipeline that continuously extracts real-world interactive SWE tasks from GitHub repositories at scale, as detailed in our paper SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software… See the full description on the dataset page: https://huggingface.co/datasets/nebius/SWE-rebench.
# download instantly
mlforge datasets pull nebius/SWE-rebench
mlforge datasets pull nebius/SWE-rebench
Dataset details
License
cc-by-4.0
Size
920 MB
Rows / images
27.9K
Classes
21
Creator
nebius
Downloads
346.8K
Source
huggingface_datasets
Updated
2025-12-23
About nebius/SWE-rebench
SWE-rebench is a large-scale dataset designed to support training and evaluation of LLM-based software engineering (SWE) agents, building upon and expanding our earlier release, SWE-bench-extra. It is constructed using a fully automated pipeline that continuously extracts real-world interactive SWE tasks from GitHub repositories at scale, as detailed in our paper SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents. The dataset currently comprises over 21,000 issue–pull request pairs from 3,400+ Python repositories, each validated for correctness through automated environment setup and test execution. A curated subset of these tasks also forms the basis of our continuously updated SWE-rebench leaderboard. SWE-rebench builds upon and extends the methodology of SWE-bench by incorporating several key enhancements detailed in our paper, including: