MigrationBench
1. ๐ Overview
๐ค MigrationBench
is a large-scale code migration benchmark dataset at the repository level,
across multiple programming languages.
- Current and initial release includes
java 8 repositories with the maven build system, as of May 2025.
2. Datasets
There are three datasets in ๐ค MigrationBench:
- All repositories included in the datasets are available on GitHub, under the
MIT or Apache-2.0 license.
3. Metadata
Metadata is provided in the csv file for each dataset:
repo (str): The original repo URL without the https://github.com/ prefix
base_commit (str): Base commit id
- At this commit with
java 8 and maven, the repository is able to (1) compile and (2) pass existing unit tests and integration tests if any
- It is the starting point for code migration from
java 8
num_java_files (int): Number of *.java files in the repository at base_commit, similarly for all other num_* columns
num_loc (int): Lines of code for the repository
num_pom_xml (int): Number of modules (pom.xml files) in the repository
num_src_test_java_files (int): Number of *.java files in the dedicated src/test/ directory
num_test_cases (int): Number of test cases, based on running the mvn -f test . command in the root directory
- Non negative values indicate number of test cases is parsed correctly from the output
- Negative values means it's unable to parse the output:
[INFO] Results: (-2) or [INFO] Tests run: (-1) regex is missing
license (str): The license of the repository, either MIT or Apache2.0 for the whole dataset
3.1 Parsing Java Test Cases
Here is a sample maven -f test . command output to get valid num_test_cases:
...
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
...
4. ๐ Citation
@misc{liu2025migrationbenchrepositorylevelcodemigration,
title={MigrationBench: Repository-Level Code Migration Benchmark from Java 8},
author={Linbo Liu and Xinle Liu and Qiang Zhou and Lin Chen and Yihan Liu and Hoan Nguyen and Behrooz Omidvar-Tehrani and Xi Shen and Jun Huan and Omer Tripp and Anoop Deoras},
year={2025},
eprint={2505.09569},
archivePrefix={arXiv},
primaryClass={cs.SE},
url={https://arxiv.org/abs/2505.09569},
}