Bugs4Q: A Benchmark of Existing Bugs to Enable Controlled Testing and Debugging Studies for Quantum Programs
Pengzhan Zhao, Miao Zhongtao, Lan Shuhan, Jianjun Zhao · Zenodo (CERN European Organization for Nuclear Research) · 2023
Realistic benchmarks of reproducible bugs and fixes are vital to good experimental evaluation of debugging and testing approaches. Bugs4Q is a benchmark of forty-two real, manually validated Qiskit bugs from three popular platforms (GitHub, StackOverflow, and Stack Exchange) in programming, supplemented with test cases to reproduce buggy behaviors. Bugs4Q Database allows users to access the bugs we collected directly. Bugs4Q Framework provides interfaces for accessing the buggy and fixed versions of the Qiskit programs and executing the corresponding source code and unit tests, facilitating reproducible empirical studies and comparisons of Qiskit program debugging and testing tools.