A Hybrid Knowledge and Transformer-Based Model for Event Detection with Automatic Self-Attention Threshold, Layer and Head Selection
Thierry Desot, Orphée De Clercq, Véronique Hoste · 2022
Event and argument role detection are frequently conceived as separate tasks.In this work we conceive both processes as one task in a hybrid event detection approach.Its main component is based on automatic keyword extraction (AKE) using the self-attention mechanism of a BERT transformer model.As a bottleneck for AKE is defining the threshold of the attention values, we propose a novel method for automatic self-attention threshold selection.It is fueled by core event information, or simply the verb and its arguments as the backbone of an event.These are outputted by a knowledge-based syntactic parser.In a second step the event core is enriched with other semantically salient words provided by the transformer model.Furthermore, we propose an automatic self-attention layer and head selection mechanism, by analyzing which self-attention cells in the BERT transformer contribute most to the hybrid event detection and which linguistic tasks they represent.This approach was integrated in a pipeline event extraction approach and outperforms three state of the art multi-task event extraction methods.