Dynamic Region-Aware Fine-Tuning: Enhancing Geographic Inclusivity in Large Language Models
K. Shantha Kumari, S.Udhaya Shree, C. Calarany · 2024
Large Language Models (LLMs) like GPT-3, BERT etc., have revolutionized natural language processing tasks across various domains. However, there exists a major challenge in LLMs, and it is Geographic bias. This bias illustrates the overrepresentation of Data only from certain parts of the world. This leads to digital exclusion of other regions and impacts the fairness, cultural sensitivity, and global applicability of AI systems. Traditional mitigation approaches, like data augmentation and bias-aware loss functions, often lack adaptability during fine-tuning. To address these gaps, a novel approach - Dynamic Region-Aware Fine-Tuning (DRAFT)is proposed. This method combines region-specific embeddings with dynamic sampling to reduce geographic bias during fine-tuning. By adjusting sampling probabilities based on region representation and embedding geographical context, DRAFT effectively reduces geographic bias. Experimental results indicate a 20% improvement in geographic diversity (Geographical Inclusion Index) and a decrease in bias (KL Divergence) without compromising accuracy. DRAFT thus offers a scalable solution for enhancing inclusivity and fairness in LLMs across diverse global regions.