SroX: Accelerating Hidden Weight Creation in Neural Networks via Spectral Relaxation and X-Iteration
Harshit Agarwal, Parth Middha, Ayush Thakur, Sofia Singh, Shipra Saraswat, Neha Bhatia · 2024
This paper proposes SroX, a novel approach for accelerating Hidden Weight creation in neural networks, which is a critical component of neural network training that can significantly impact the overall efficiency of the training process. SroX leverages spectral relaxation and X-Iteration to reduce the computational time required for Hidden Weight creation, enabling faster training times without sacrificing accuracy. Our approach addresses the long-standing challenge of efficient Hidden Weight creation, which has been a major bottleneck in neural network training. We demonstrate the efficacy of SroX through extensive experiments on benchmark datasets, including MNIST, CIFAR-10, and ImageNet, where SroX outperforms existing methods, including Random Projection and Gradient-Based Optimization, in terms of training time and accuracy. The proposed approach provides a promising solution for large-scale neural network applications, where efficient training is crucial, and has the potential to revolutionize the field of neural network research and applications, driving innovation and breakthroughs in areas such as computer vision, natural language processing, and beyond. By providing a faster and more efficient way to create Hidden Weights, SroX can enable researchers and practitioners to tackle complex problems with unprecedented speed and efficiency, leading to significant advancements in the field of artificial intelligence.