C3: Code Clone-based Identification of Duplicated Component
Anonymous · Zenodo (CERN European Organization for Nuclear Research) · 2022
The source code of C3 consists of two sections. The first section is used to pre-process the raw dataset. The second section is used to identify cloned components in software. The model runs on python3. In the first part: We first get all building files to obtain the composition information of components in software systems. Note that: different systems may adopt different forms as their building files. For example, in our study, our dataset uses ''CmakeList'' to build their project and Linux adopts ''MakeFile'' as their building files. Therefore, if you want to detect cloned components in Linux, you need to change the related code for suitable the grammar that Linux's building files use. After analyzing the building files, we know the source code in different components and collect the components by zipping the corresponding source code. We use SourcererCC, which is an existing clone detection tool (https://github.com/Mondego/SourcererCC) to detect file-level clones in the system. This tool can detect clones simultaneously for multiple projects。 After detecting the cloned files, we will get two useful files generated by SourcererCC. The first file is: files-stats-0.stats In this file, SourcererCC will provide an ID for all source files in the system. the form is like: 1,1,"XXX/XXX/XXX/XXX/tools/XXX/stub.c","d41d8cd98f00b204e9800998ecf8427e",0,1,1,0 1,18,"XXX/XXX/XXX/XXX/tools/XXX/LLT_UserConfig.c","6d057b39787dafd44717fca6e47578e5",6576,219,184,26 3,22,"XXX/XXX/XXX/XXX//mpu/XXX/sa_omc_ftpc.c","87e843612424455f213b42898b1980cb",63965,1682,1462,1182 The first number represents the component ID and the second number denotes the file ID in this component. For example, the second line represents the 18th source file in Component 1 is ''LLT_UserConfig.c'' and this file and the ''stub.c'' are both the source code in the same component. The third line denotes the 22th source file in Component 3. Note that: the file ID is sequential. The second useful file generated by SourcererCC notes the clone result. The form is: 1,2,4,28 1,4,8,59 ... The first line means the second files in Component 1 is similar to the 28th source file in Component 4. Based on file-level clone results, we calculate the similarity (i.e., clone ratio) between components. We use 11 metrics to describe a component, including: cmpt_id: component ID cmpt_name: component name cmpt_file_num: the number of source files in a component cmpt_code_line: the line of code in each source file in this component clone_cmpt_with_id: (list) the ID of a component that is similar to this component cmpt_clone_file_tuple: records the component ID and source file ID of other components that have similar source files to the source files in this component. [(component ID, file ID), ...] cross_cmpt_clone_file_in_id: the file ID that are similar to the source code of this component cross_cmpt_clone_ratio: the average cross clone ratio cross_cmpt_clone_ratio_with_each_cloned_cmpt: the clone ratio (similarity) with other components (Here other components refer to the components that have similar code to this component.) within_clone_file_in_id: within clone files in this components (i.e., the two source files in a file-level clone pair are both in this component.) In the second part: We first achieve the edges from the cloning relationships. You can run ''python generate_model_input.py para'' (e.g., python generate_model_input.py 1) to genrate the edges. the parameter set 1 which means the k value is set to 0.1. So para can be set from 0 to 10 (int). We build a community network to describe the cloning relationships between components and adopt a community detection algorithm to obtain component-level clone groups. You can run ''python clone_cluster.py para'' (e.g., python clone_cluster.py 5) to get the component-level clone groups under different parameter values. The result form is like: [652, 781, 782, 780, 784, 783, 805, 688, 689, 691, 955, 700, 705, 839, 715, 716, 718, 848, 853] [773, 687, 721, 754, 659, 732, 638, 671]. ... A list represents a clone group, and the IDs in a list are the component IDs.