Zero-Shot Document Classification Using Pretrained Models
Madhulika Yarlagadda, Susrutha Ettimalla, Bhanu Sri Davuluri · 2024
The research introduces an innovative method for document classification in Natural Language Processing (NLP) by employing Pretrained Bidirectional Encoder Representations from Transformers (BERT) within a zero-shot learning framework. Contrary to conventional methods that necessitate labeled training data for each class, the approach utilizes contextual embeddings from pretrained transformer models to categorize documents into multiple classes without direct training on those categories. The bidirectional characteristics of transformers allow the model to grasp intricate semantic relationships within documents, ensuring robust adaptability to a wide array of document genres. Experimental findings on benchmark datasets underscore the efficacy of the zero-shot method in comparison to conventional supervised techniques. Specifically, the S-BERT model demonstrates superior performance in zero-shot document classification, exhibiting elevated accuracy, precision, recall, and F1-scores. This underscores its potential applicability in news categorization and content analysis without the need for finetuning or specific task adjustments. Overall, harnessing pretrained transformer models within a zero-shot framework presents a promising direction for developing scalable and flexible document classification systems spanning diverse domains.