Optimizing Text Classification Using Techniques AdaBoost Ensemble with Decision Tree Algorithm
Marnis Nasution, Ibnu Rasyid Munthe, Fitri Aini Nasution, Sarjon Defit · CogITo Smart Journal · 2025
This study presents an optimized text classification framework combining AdaBoost ensemble techniques with Decision Tree algorithms (ID3, C4.5, CART) to address critical challenges in small dataset scenarios (n=795 Indonesian-language reviews). Employing rigorous five-fold stratified cross-validation (random seed=42), we implemented a comprehensive preprocessing pipeline including case normalization, language-specific stemming, and TF-IDF feature extraction. The ensemble model utilized 50 AdaBoost iterations with a learning rate of 1.0, evaluated through multiple performance metrics while accounting for class imbalance effects. Key results demonstrate significant performance enhancements, with the C4.5+AdaBoost configuration achieving 96.72% accuracy (±0.88), representing a 10.6 percentage point improvement over the base C4.5 algorithm. The ensemble approach particularly improved minority class identification, boosting positive sentiment classification F1-scores by 0.28 points while maintaining exceptional neutral sentiment detection (F1-score 0.99±0.00). Comparative analysis revealed consistent advantages across all Decision Tree variants, with accuracy improvements of 18.6% for ID3, 10.6% for C4.5, and 14.2% for CART, alongside reduced performance variance (62-73% decrease). While these findings validate AdaBoost's effectiveness for enhancing Decision Tree stability in small-scale text classification, the study acknowledges limitations regarding sample size constraints and language specificity. The research contributes practical methodologies for sentiment analysis applications while emphasizing the need for validation on larger, more diverse datasets. Future work should explore comparative benchmarking against transformer architectures. Advanced feature representation techniques and multilingual generalization testing. This work provides a reproducible framework for developing robust, ensemble-based text classification systems in resource-constrained scenarios.