Improving Multihead Finite State Machine With Transformer Neural Network
Ulyana Pavlova, Valerii Boldakov · 2022 IEEE International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON) · 2022
This paper considers the integration of a neural network into a multi-head automaton designed to recognize multilinear sequences. The main problem of the automaton is a low quality for recognition of sequences that differ from multilinear pattern. This problem is proposed to be solved using neural networks. In this paper a novel technique based on transformer architecture is proposed for detection of the outliers in the multilinear sequence. BERT-like architecture was chosen because of an ability to model long interval dependencies, compared to LSTM or CNN models. These models were trained for detection of irregularities in multilinear sequences. Suggested model was validated on different randomly created sequences.