Software and Infrastructure Log-Based Framework for Identifying the Causes of System Faults
Noriko Hanakawa, Masaki Obana · 2018
Recently, computer systems have become increasingly complex, involving a wide range of infrastructure and software technologies in the same system. Because these complex technologies affect each other, it is difficult to identify the causes of system faults while the system is running. Faults caused by a mix of software and infrastructure issues are particularly difficult for software engineers to resolve. We therefore propose a framework for identifying the causes of system faults that targets both software and infrastructure problems. The aim of this research is to identify faults that are related to software and/or infrastructure. The framework is based on system logs, including both software and infrastructure logs. When a fault occurs while a system is running, software and infrastructure logs are first collected and then compared with a large dataset of past logs to rapidly identify the cause of the fault. After that, debug mode level logs are collected in order to identify proper bugs of source code level and/or setting values of infrastructure configuration file level. The framework is also applied to two examples of real system faults. We find that the first fault was caused by software bugs, but the second was caused by both software bugs and infrastructure problems. We have also implemented a tracing and replay tool based on the framework. We tried to detect the true causes of 17 system faults that were randomly picked up past log dataset. 96% system logs in the 17 faults occurrence operation patterns were detected the differences between fault and normal cases. Then the two concrete session error faults occurred by two causes, infrastructure problems and software bugs. In this way, the framework will help software engineers to identify the causes of faults even when they find it difficult to separate software and infrastructure issues.