Winning ClimateCheck: A Multi-Stage System with BM25, BGE-Reranker Ensembles, and LLM-based Analysis for Scientific Abstract Retrieval
Junjun Wang, Kunlong Chen, Zhaoqun Chen, Peng He, Zheng Wu · 2025
The ClimateCheck shared task addresses the critical challenge of grounding social media claims about climate change in scientific literature.This paper details our winning approach for solving two subtasks.For abstract retrieval, we propose a multi-stage pipeline:(1) initial candidate generation from a corpus of ∼400,000 abstracts using BM25; (2) finegrained reranking of these candidates using an ensemble of BGE-Reranker cross-encoder models, fine-tuned with a specialized training set incorporating both random and hard negative samples; and (3) final list selection based on an RRF-ensembled score.For the verification aspect, we leverage Gemini 2.5 Pro to classify the relationship between claims and the retrieved abstracts.Our system achieved first place in both subtasks.Part of the example code: https://github.com/cklcklcklckl/ climatecheck_1st_solution.