ADMEOOD: Out-of-Distribution Benchmark for Drug Property Prediction
Shuoying Wei, Yifei Guo, Xinlong Wen, Songquan Li, Lida Zhu, Rongbo Zhu · 2023
Obtaining accurate and effective information for drug molecules is a crucial and challenging task, which relies on high-quality chemical knowledge. However, chemical knowledge has been accumulated over the past 100 years from various regions, laboratories, and experimental purposes, which contains a lot of noise and inconsistency, leading to the out-of-distribution (OOD) problem. OOD may results in weak robustness and unsatisfied performance. In order to solve OOD learning problem with noise, a novel benchmark: ADMEOOD is proposed, which is a systematic OOD dataset curator and specifically designed for drug property prediction. ADMEOOD screens 27 Absorption, Distribution, Metabolism and Excretion (ADME) drug properties from Chembl and relevant literature. This paper explicitly make distinctions between two kinds of OOD data shifts: Noise Shift and Concept Conflict Drift (CCD). Overall, ADME contains 6 domain annotations combined with noise, CCD and no shifts, resulting in 18 different splits in total. ADMEOOD provides performance results on a variety of SOTA OOD models. The results demonstrate a significant difference performance between in-distribution and OOD data. Moreover, Empirical Risk Minimization and other models exhibit distinct trends in different domains and measurement types. The ADMEOOD benchmark can be accessed via https://github.com/qweasdzxc-wsy/ADMEOOD/.