Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models
Boheng Sheng, Jiacheng Yao, Meicong Zhang, Guoxiu He · 2025
Large language models (LLMs) often struggle to accurately read and comprehend extremely long texts.Current methods for improvement typically rely on splitting long contexts into fixed-length chunks.However, fixed truncation risks separating semantically relevant content, leading to ambiguity and compromising accurate understanding.To overcome this limitation, we propose a straightforward approach for dynamically separating and selecting chunks of long context, facilitating a more streamlined input for LLMs.In particular, we compute semantic similarities between adjacent sentences, using lower similarities to adaptively divide long contexts into variable-length chunks.We further train a question-aware classifier to select sensitive chunks that are critical for answering specific questions.Experimental results on both singlehop and multi-hop question-answering benchmarks show that the proposed approach consistently outperforms strong baselines.Notably, it maintains robustness across a wide range of input lengths, handling sequences of up to 256k tokens.Our datasets and