Impact of LLM Scale and Quantization on Information Extraction from Clinical Text
Alban Bornet, Abiram Sandralegar, Anthony Yazdani, Philippe Bijlenga, Douglas Teodoro · Studies in health technology and informatics · 2026
Large Language Models (LLMs) show strong potential for extracting structured information from unstructured clinical narratives. However, their adoption in healthcare is constrained by privacy requirements that necessitate local deployment, often under limited computational resources. This study systematically investigates the trade-offs between model size and numerical precision (quantization) in locally deployed, open-source LLMs applied to a clinical information extraction task. Using seven reasoning models from the Qwen3 and DeepSeek-R1 families (0.6B-70B parameters) and six quantization levels (2, 3, 4, 5, 6, and 8 bits), we evaluated performance on the classification of the modified Rankin Scale (mRS) score from 105 neurosurgery patient notes. A linear mixed-effects analysis revealed significant effects for log-scaled model size (β=-0.84, p<0.001), quantization (β=-0.12, p<0.001), and their interaction (β=0.08, p<0.001). Larger models and higher numerical precision yielded lower extraction error, but the performance plateaued at 4-5-bit precision. Our results suggest that clinically useful information extraction is achievable with moderately sized, locally deployed LLMs, supporting their feasibility for privacy-preserving healthcare applications.