Emergence of hierarchical modes from deep learning
Chan Li, Haiping Huang · Physical Review Research · 2023
Training deep neural networks in mode space rather than traditional weight space not only saves a significant (greater than 50%) training expense but also leads to more disentangled and scale-invariant emergent representations in the hierarchy, which thereby makes deep learning fast and more transparent.