A parallel circuit approach for improving the speed and generalization properties of neural networks
Kien Tuong Phan, Tomás Maul, Tuong Thuy Vu · 2015
One of the common problems of neural networks, especially those with many layers consists of their lengthy training times. We attempted to solve this problem at the algorithmic (not hardware) level, proposing a simple parallel design inspired by the parallel circuits found in the human retina. To avoid large matrix calculations, we split the original network vertically into parallel circuits and let the BP algorithm flow in each subnetwork independently. Experimental results have shown the speed advantage of the proposed approach but also pointed out that the reduction is affected by multiple dependencies. The results also suggest that parallel circuits improve the generalization ability of neural networks presumably due to automatic problem decomposition.