PAC Learnability and Information Bottleneck in Deep Learning: Tutorial and Survey
Benyamin Ghojogh, Ali Ghodsi · 2024
This is a tutorial and survey on PAC learnability and information bottleneck in deep learning. This paper first introduced PAC learnability, agnostic PAC learnability, the VC dimension, and error bounds in neural networks. It discusses that PAC learnability in deep learning does not align with the performance of deep learning in practice. Then, the paper explains the generalization of deep learning in terms of information theory and information bottleneck.