Discovering Cloud Operation History through Log Analysis
Mitsunari Kobayashi, Yosuke Himura, Yoshiko Yasuda · 2017
Management costs in private clouds will be promisingly reduced by reviewing `operation history,' which is defined as a holistic view of past operation executions. Operation history provides insights into breakdown of operations: the breakdown clarifies cost-dominant operations to be improved and repetitive ones to be automated. Towards obtaining the operation history, a conventional approach relying on manual investigation is time-consuming, and another relying on agent-based monitoring is not often acceptable in sensitive mission-critical enterprise clouds. Different from these approaches, our idea is to discover the operation history by automatically analyzing `system logs' that are easily accessible even in sensitive clouds. Since system logs contain only low-level debugging messages about programmatic events without direct contexts about operations, the challenge is to recover high-level operational contexts from low-level system logs. To address this challenge, we develop a method that first abstracts system logs using a pre-defined event sequence model, and then maps the abstracted events to high-level individual operations-this mapping between different contextual levels is achieved by using complementary cross-cloud reference data. Evaluation of an implementation revealed that this method reduces the time taken to discover the history by 99.9% compared to a conventional approach while achieving up to 95% correctness.