TELIN: Table Entity LINker for Extracting Leaderboards from Machine Learning Publications
Sean T. Yang, Chris Tensmeyer, Curtis Wigington · 2022
Tracking state-of-the-art (SOTA) results in machine learning studies is challenging due to high publication volume.Existing methods for creating leaderboards in scientific documents require significant human supervision or rely on scarcely available L A T E X source files.We propose Table Entity LINker (TELIN), a framework which extracts (task, model, dataset, metric) quadruples from collections of scientific publications in PDF format.TELIN identifies scientific named entities, constructs a knowledge base, and leverages human feedback to iteratively refine automatic extractions.TELIN identifies and prioritizes uncertain and impactful entities for human review to create a cascade effect for leaderboard completion.We show that TELIN is competitive with the SOTA but requires much less human annotation.