Detection of test smells with basic language analysis methods and its evaluation
Florian C. Maier, Michael Felderer · 2023
Similar to the concept of "code smells", so-called "test smells" describe situations in test code that possibly indicate a deeper problem and require refactoring measures. The elimination of test smells ensures, among other things, that tests are easy to read, easier to maintain and less prone to errors. To automate the detection of test smells, we created the tool "SniffTest" (STest), which detects five types of test smells in JUnit tests, namely Anonymous Test, Long Test, Conditional Test Logic, Assertion Roulette, and Rotten Green Test. STest applies basic natural language processing methods, namely regular expressions and part-of-speech tagging. These methods are mostly programming language independent and can be easily adapted to other languages and frameworks. To measure the accuracy of the tool, we created a labelled dataset containing 854 JUnit test methods and 106 helper methods from eight large open source GitHub repositories, where we manually annotated the test smells. The tool showed an accuracy of 87 to 100% in detecting the different test smells. Both the tool and the dataset are made available as open source.