Commonly used software tools produce conflicting and overly-optimistic AUPRC values
Wenyu Chen, Miao Chen, Zhenghao Zhang, Cathy Sin‐Hang Fung, Ran Wang, Yizhen Chen, Qian Yan, Lixin Cheng, Kevin Yuk-Lap Yip, Stephen Kwok‐Wing Tsui, Qin Cao · Genome biology · 2024
The precision-recall curve (PRC) and the area under the precision-recall curve (AUPRC) are useful for quantifying classification performance. They are commonly used in situations with imbalanced classes, such as cancer diagnosis and cell type annotation. We evaluate 10 popular tools for plotting PRC and computing AUPRC, which were collectively used in more than 3000 published studies. We find the AUPRC values computed by the tools rank classifiers differently and some tools produce overly-optimistic results.