Expert Evaluation Protocol for Evidence-Graded Knowledge Graph Ranking Quality
Rogier Meulenaar · Zenodo (CERN European Organization for Nuclear Research) · 2026
Paper 10 in the Prioris research series. Specifies a protocol for expert evaluation of edge ranking quality across evidence levels. Defines evaluation dimensions, statistical analysis methods (Kendall's tau_b, Krippendorff's alpha), and software specifications. Includes pilot results (tau=0.58, alpha=0.71) from 3 experts on 20 edges. Part of the Evidence-Graded Knowledge Graphs research programme (Paper 0: 10.5281/zenodo.19024611).