Towards Compact and Explainable Deep Belief Networks
Jan Bronec, Iveta Mrázová · 2024
Vast resources available in the data domain and hardware provided impact impressive achievements in generative AI. A growing interest in IoT and mobile/edge computing is, however, pushing research towards heavy architecture optimization and easy interpretability. Our work investigates viable opportunities to extract knowledge from Deep Belief Networks and explain it understandably. To initiate compact knowledge representation, we specify a feasible training and pruning strategy for the networks. The so-called confidence rules constitute an elegant means to express quantitative reasoning performed by the network. The paper introduces a new approach to extracting confidence rules from the networks that allows for an enhanced accuracy of the inference. Compared to rival rule extraction techniques, supporting experiments confirm a significant improvement in inference accuracy obtained with less than one-fifth of the original network weights.