Construction of a recommendation system based on spin-orbit torque binary stochastic neurons
Xiaozhou Ye, Wei Duan, Zhen Cao, Kaiyuan Wang, Long You · Chinese Science Bulletin (Chinese Version) · 2024
In recent years, with the rapid development of artificial intelligence and deep learning, a variety of artificial neural networks have emerged. Among these, stochastic generative neural networks, exemplified by Boltzmann machines and restricted Boltzmann machines (RBMs), have garnered significant attention due to their powerful capabilities in feature learning and data reconstruction. These networks are widely employed in unsupervised classification and recommendation tasks. Despite the robust functionality of these neural network models, their performance is currently constrained by the hardware totally based on complementary metal-oxide semiconductor (CMOS) digital circuits. When implemented in digital circuits, the stochastic sampling process in RBM neural networks incurs substantial area and power costs, primarily due to the generation of random numbers and the execution of non-linear activation functions. However, the advent of spintronic devices such as spin-orbit torque magnetic tunnel junctions (SOT-MTJs), which leverage spin stochastic dynamics to achieve intrinsic non-linearity, true randomness and tunable probabilities, offers a promising alternative for improving performance. To verify the feasibility of constructing RBM neural networks with binary stochastic neurons (BSNs) based on SOT-MTJs, this work embarked on several key steps. First, we established an electrical model of the SOT-BSN utilizing the validated SOT-MTJ Verilog-A model and 65 nm CMOS transistors and obtained the characteristic of the switching probability controlled by input voltages through SPICE simulations. The characteristic curve closely aligns with the sigmoid function which plays a weighty role in the RBM stochastic sampling process. The simulation results indicate that SOT-BSNs can effectively perform the probabilistic sampling operations required by RBM networks with each operation consuming only 17.7 fJ and taking 10 ns. Subsequently, the fitted curve of the switching characteristics was applied to a recommendation system based on RBM, and we performed training on a movie rating dataset to enable the system to capture user preferences and make recommendations. During training, the RBM network learned from the dataset by reconstructing user rating patterns, thus enabling it to predict unrated movies. The results were highly promising, with the recommendation system achieving a prediction accuracy of 96.08%. This demonstrates the potential of SOT-BSNs as efficient hardware accelerators for RBM neural networks, offering a significant improvement in computational efficiency over traditional digital circuits implementations. Lastly, this paper investigated the impact of non-ideal factors of SOT-BSN devices on the performance of the RBM system. We particularly focused on device-to-device (D2D) variation simulated by introducing a Gaussian distribution into the device parameters. The simulation revealed how such variations influence the overall accuracy of the RBM system, which offers some guidance for practical hardware implementation.