Attention Mechanism Based Technical Phrase Extraction for Patent Text
Xin Jiang, Peng Zhou, Shu Guo Zhao, Yufeng Guo · 2023
Technological innovation is becoming one of the critical factors in promoting social development all over the world. The vigorous development of patent applications in recent years provides an opportunity to reveal the inherent laws of innovation, but it also puts forward higher requirements for patent mining technology. An essential step in patent text mining is to build a technical portrait for each patent, that is, to identify the technical phrases involved, which can summarize and represent the patent from a technical perspective. Previous technical phrase extraction methods thoroughly used technical phrases' characteristics and the relationship between technical phrases. Regarding our observations, the relationship between patent texts and technical phrases is also essential. Specifically, critical technical phrases are more relevant to the patent text and can be discovered by the attention mechanism. Motivated by this, we propose an unsupervised technical phrase extraction method based on the attention mechanism named UTESC. Self-attention captures the importance of technical phrases in sentences, and cross-attention captures the relevance between technical phrases and patents. Extensive experiments and algorithm comparisons on patent datasets have proven the effectiveness of our algorithm.