Balanced Knowledge Transfer in MTTL-ClinicalBERT: A Symmetrical Multi-Task Learning Framework for Clinical Text Classification
Qun Zhang, Shiyang Chen, Wenhe Liu · Symmetry · 2025
Clinical text classification presents significant challenges in healthcare informatics due to inherent asymmetries in domain-specific terminology, knowledge distribution across specialties, and imbalanced data availability. We introduce MTTL-ClinicalBERT, a symmetrical multi-task transfer learning framework that harmonizes knowledge sharing across diverse medical specialties while maintaining balanced performance. Our approach addresses the fundamental problem of symmetry in knowledge transfer through three innovative components: (1) an adaptive knowledge distillation mechanism that creates symmetrical information flow between related medical domains while preventing negative transfer; (2) a bidirectional hierarchical attention architecture that establishes symmetry between local terminology analysis and global contextual understanding; and (3) a dynamic task-weighting strategy that maintains equilibrium in the learning process across asymmetrically distributed medical specialties. Extensive experiments on the MTSamples dataset demonstrate that our symmetrical approach consistently outperforms asymmetric baselines, achieving average improvements of 7.2% in accuracy and 6.8% in F1-score across five major specialties. The framework’s knowledge transfer patterns reveal a symmetric similarity matrix between specialties, with strongest bidirectional connections between cardiovascular/pulmonary and surgical domains (similarity score 0.83). Our model demonstrates remarkable stability and balance in low-resource scenarios, maintaining over 85% classification accuracy with only 30% of training data. The proposed framework not only advances clinical text classification through its symmetrical design but also provides valuable insights into balanced information sharing between different medical domains, with broader implications for symmetrical knowledge transfer in multi-domain machine learning systems.