A Novel Semantic-Enhanced Text Graph Representation Learning Approach through Transformer Paradigm
Tham Vo · Cybernetics & Systems · 2022
Among common tasks in natural language processing (NLP) domain, text classification is considered as an important primitive task which is widely applied in multiple disciplines. Recent advanced deep learning-based architectures such as sequence-to-sequence (seq2seq) with attention mechanism have demonstrated remarkable improvements in multiple NLP’s tasks, including classification. However, recent seq2seq-based models still encounter challenges related to the limitation in effectively capturing long-range dependent relationships between words in a text corpus. Recent integrated graph neural network and textual graph transformer (TGT)-based models have demonstrated significant improvements in preserving the structural n-hop co-occurring relationships between words in a given text corpus. However, these models still suffer problems related to the thorough considerations on the sequential and contextual relations of words within a single document’s graph. To meet these challenges, in this article we proposed a novel semantic-enhanced graph transformer-based textual representation learning approach, called as: SemTGT. Our proposed SemTGT can support to effectively learn both local rich-contextual and global long-range structural latent representations of texts for leveraging the performance of classification task. Extensive experiments in standard datasets demonstrate the effectiveness of our proposed SemTGT model in comparing with recent seq2seq-based and textual graph embedding-based baselines.