Replication Package for : AML: A novel approach towards critical analysis of log parsing strategies
Anonymous · Zenodo (CERN European Organization for Nuclear Research) · 2023
This replication package has been prepared to support further investigation and validation of our study on AML metric to measure the accuracy of log parsing techniques. It contains detailed information about log parsing results and the the calculation results for each log parser evaluated in our research. The package is organized into separate zipped folders. The package is organized as follows: Parsing results: There are separate folders for each log parsing tool, such as ULP, Drain, etc. Inside each folder, you will find individual CSV files containing the parsing results for each dataset analyzed in our study. These files are named as "Android_2k.log_structured.csv", "Apache_2k.log_structured.csv", and so on. Each row represents a single log entry, with the columns providing various details about the log entry (header and message). The column EventTemplate represents the parsed log event template. AML calculation : There are separate folders for each log parser. Inside each folder, you will find individual CSV files containing the parsing results for each dataset analyzed in our study. These files are named as "AML_ _final.csv, where is the name of the Dataset parsed. each file contains four columns: Template, Omission Error, Commission Error, and ICSI. Each row represents a specific log template, along with the associated score for evaluating log parsing performance on the template level. at the end of the file, the overall AML score if transcribed. (Note that: Iplom timed out for Android) Overall, here are the AML results for all the datasets and log parsers: System HDFS Apache Android Windows Mac OpenStack Linux OpenSSH Thunderbird Spark Hadoop Zookeeper BGL HealthApp HPC Proxifier Average ULP 0.5681 0.3333 0.4633 0.3866 0.4435 0.2800 -0.2802 0.0690 0.4103 0.3077 0.1684 0.6508 0.2542 0.7500 0.4774 0.4330 0.3572 Drain 0.7055 0.3333 0.6605 0.3919 0.1709 0.0088 -0.2478 0.2439 0.1871 0.2500 0.1361 0.3913 0.1443 0.0871 0.3433 0.3730 0.2612 SHISO 0.7500 0.3333 0.4313 0.2790 0.1455 0.1111 0.2274 0.1735 0.1718 0.2241 0.1194 0.2333 0.0947 0.1681 0.0917 0.4612 0.2510 AEL 0.6663 0.3333 0.4976 0.3151 0.1356 0.1868 -0.1948 0.1731 0.1605 0.1605 0.1187 0.3205 0.1451 0.0694 0.4091 0.4244 0.2451 Iplom 0.7100 0.3300 -- 0.2700 0.1400 0.2100 -0.1900 0.1600 0.1800 0.2500 0.1400 0.4000 0.1300 0.1400 0.3400 0.5532 0.2352 Spell 0.7500 0.3333 0.7294 0.3312 0.1264 0.0123 -0.1964 0.2000 0.1399 0.1605 0.0977 0.3049 0.0527 0.1034 0.3905 0.0585 0.2246 Lenma 0.7055 0.3333 0.6508 0.2976 0.1246 0.1155 -0.2550 0.1860 0.1999 0.1074 0.1088 0.3292 0.0613 0.0282 0.1117 0.0910 0.1997 LogMine 0.6875 0.3333 0.3083 0.3247 0.1671 0.0097 0.2473 0.0107 0.1631 0.0741 0.1186 0.0727 0.0266 0.0630 0.0528 0.0222 0.1676 LKE 0.7500 0.3333 0.7563 0.2877 0.0000 0.0242 -0.0767 0.0239 0.0640 0.0000 0.0506 0.1923 0.0200 0.1709 0.0000 0.0998 0.1685 SLCT 0.2166 0.1429 0.7000 0.0506 0.0870 0.2679 0.0310 0.1260 0.0599 0.1167 0.0079 0.1467 0.0661 0.0361 0.2702 0.1654 0.1557 Logsig 0.2760 0.1820 0.3450 0.1590 0.0720 0.2470 0.0400 0.1700 0.0240 0.0620 0.0200 0.1820 0.0700 0.0490 0.0810 0.3447 0.1452 Molfi 0.0000 0.0000 0.1703 0.1770 0.0676 0.0018 0.2275 0.0612 0.1368 0.0068 0.0824 0.1585 0.0511 0.2477 0.1634 0.0000 0.0970 Logram 0.0141 0.0009 0.3290 0.0806 0.1133 0.0000 0.0303 0.1172 0.0533 0.0172 0.0303 0.2240 0.0458 0.0288 0.2871 0.0000 0.0857 LogCluster 0.0056 0.0000 0.1786 0.2414 0.0846 0.0023 -0.1906 0.0282 0.1256 0.0267 0.0619 0.1545 0.0309 0.0414 0.1170 0.0000 0.0568