EvalAssist: Insights on Task-Specific Evaluations and AI-Assisted Judgment Strategy Preferences
Zahra Ashktorab, Michael J Desmond, Qian Pan, James M. Johnson, Martín Santillán Cooper, Elizabeth Daly, Rahul Nair, Tejaswini Pedapati, Hyo Jin Do, Werner Geyer · 2025
User flow diagram for EvalAssist in the direct assessment evaluation, illustrating criteria definition, test data input, annotation, AI evaluator selection, result review, iterative adjustments, and criteria export for dataset-wide evaluation via SDK.