Evaluation of Deep Neural Network Quality by CleanAI Coverage Metrics Library
Abdul Hannan Ayubi, Furkan Taskin, Osman Çağlar, Sergen Aşik, Cem Bağlum, Uğur Yayan · 2023
The increasing use of AI-based systems in critical domains necessitates the assessment of their quality and reliability. However, there is a lack of readily available tools for analyzing metrics related to the structure, security, safety, and quality control of Deep Neural Network (DNN) models. To address this gap, CleanAI is introduced as a white-box testing library that utilizes coverage metrics to evaluate the structural analysis, quality, and reliability parameters of DNN models. The library incorporates eleven coverage test methods and enables developers to perform essential model analyses and generate output reports. By using CleanAI, developers can effectively evaluate the quality of DNNs and make informed decisions. The library's effectiveness was demonstrated through testing popular models like ResNet50, providing insights into neuron coverage, threshold value coverage, and other coverage metrics. The adoption of CleanAI can significantly evaluate the quality of AI-based systems and result in time and labor cost savings for companies and developers in the industry.