Evaluating Column Type Annotation Models and Benchmarks
Amir Babamahmoudi, Davood Rafiei, Mário A. Nascimento · 2025
Column Type Annotation (CTA) is the process of assigning semantic types to the columns of a table. Models and approaches for CTA are typically evaluated on distinct datasets, making benchmarking and comparisons not trivial. In this paper, we identify major issues in commonly-used datasets for CTA and explore their impact on the evaluation process and reported results. Drawing from our analysis, we propose changes towards more robust and unbiased benchmarking practices.