Linguistically-Informed Sparse Attention with Adaptive Knowledge Graph Integration for Multi-Task Text Classification and Sentiment Analysis
Yaping Zhang, Mengqin Sun, Shaoying Yang · 2025
This paper proposes a novel multi-task learning framework that synergizes linguistically-guided sparse attention with adaptive knowledge graph integration to advance text classification and sentiment analysis. Our approach introduces a hierarchical sparse attention mechanism that automatically derives optimal sparsity patterns from dependency parse trees, mimicking Bayesian sparsity constraints without manual feature engineering, combined with an evolving graph attention network featuring gated incremental updates for dynamic knowledge embedding refinement. The framework employs predictive coding-inspired dynamic regularization to balance linguistic and task-specific demands. Comprehensive experiments demonstrate state-of-the-art performance, achieving 94.8% accuracy on SST-2 (outperforming BERT-base by 2.3%), 77.5% F1 on aspect-based sentiment analysis (SemEval-2014 Task 4), while maintaining 2.1xfaster inference through 52% attention sparsity and reducing memory consumption by 47% compared to dense transformers. The results validate that integrating linguistic priors with adaptive knowledge graphs creates an efficient paradigm for multi-task NLP.