An efficient SOM-based pre-processing to improve the discovery of frequent patterns in alarm logs.
Françoise Fessant, Fabrice Clérot · DMIN · 2006
Abstract - We describe a pre-processing technique for mining a telecommunication alarm log for frequent temporal patterns. The method consists in extracting relevant subsets from the initial log with the aim of discovering frequent patterns more accurately. In a first step, the alarm types presenting the same temporal behaviour are clustered with a self organizing map. Then, log areas which are rich in alarms of each cluster are regrouped in subsets which are subsequently exhaustively searched for frequent patterns. We demonstrate the efficiency of our pre-processing method through experiments on an actual alarm log from an ATM network. Keywords: Self-organizing map, chronicle recognition, telecommunications network supervision. 1 Introduction Telecommunications networks are growing in size and complexity, which means that a constantly increasing volume of notifications must be handled by the management system. Most of this information is produced spontaneously by equipments (e.g. status change and dysfunction detection) and this message flow must be preprocessed to make an effective management possible. Filters based on a per-notification basis fail to perform an adequate information pre-processing required by human operators or by management application software which are not able to process such amount of events. A pre-processing stage must “thin” this information stream by suppressing redundant notifications and/or by aggregating relevant ones. Numerical time constraints must also be taken into account since time information is apropos for the telecommunications alarm propagation. Many works deal with different approaches and propose more or less complex intelligent filtering: one can use some efficient rule-based languages [1], and/or object-based techniques [2]. More specific techniques are devoted to capture time constraints between alarms [3], [4]. In any case, the problem of expertise acquisition remains the same: how to feed the filtering system? Which aggregation rules are relevant? A way to filter the information flow is to exploit the logs collected from telecommunications equipment. We have developed a tool called FACE (Frequency Analyzer for Chronicle Extraction) which performs a frequency-based analysis in order to extract “frequent patterns” from the logs. We only suppose that all the events are time-stamped. The searched patterns are sets of event patterns with time constraints between them (these sets are called “chronicle models”) and the frequency criterion is defined as a user-defined minimal frequency threshold [5]. Identifying the most frequent chronicle models is relevant to reduce the number of alarms displayed to the operator: if a chronicle corresponds to a dysfunction, the corresponding set of alarms is aggregated before being displayed to the human operator; if not, the corresponding set of alarms is filtered. At the moment, we do not use any extra knowledge about the domain; the rule qualification (aggregation or filtering) is performed by an expert at the end of the discovering process. Real experiments on telecommunications data show that most of the discovered chronicles are relevant and some of them have a real benefit for the experts. However, the chronicle process implemented in FACE, based on the exhaustive exploration of the chronicle instances in the alarm log, is very memory-space consuming and the main factor of this explosion is the size of the processed event logs. To deal with that problem in the current implementation of FACE, the operational experts manually select some alarms types and/or some time periods in the alarms logs in order to extract a more manageable sublog to be processed by the tool. The purpose of this communication is to describe and to evaluate a pre-processing method to automatically extract relevant sublogs from an initial alarm log to simplify the use of the software and alleviate the memory saturation effect. The pre-processing stage we propose can be decomposed in two main steps. Firstly we group together the alarm types which exhibit the same temporal behavior. We introduce an original