RefactorScore: Evaluating Refactor Prone Code
Kevin Jesse, Christoph Kuhmuench, Anand Ashok Sawant · IEEE Transactions on Software Engineering · 2023
We proposeRefactorScore, an automatic evaluation metric for code.RefactorScorecomputes the number of refactor prone locations on each token in a candidate file and maps the occurrences into a quantile to produce a score.RefactorScoreis evaluated across 61,735 commits and uses a model calledRefactorBERTtrained to predict refactors on 1,111,246 commits. Finally, we validateRefactorScoreon a set of industry leading projects providing each with aRefactorScore. We calibrateRefactorScore's detection of low quality code with human developers through a human subject study.RefactorBERT, the model driving the scoring mechanism, is capable of predicting defects and refactors predicted byRefDiff 2.0. To our knowledge, our approach, coupled with the use of large scale data for training and validated with human developers, is the first code quality scoring metric of its kind.