Parameter Design Approaches based on AI Techniques for Transformer Neural Network Optimization
Gurpreet Singh, Mansi Chaudhary, Jaspreet Singh, Nikita Madaan · 2023
The Transformer neural network is a magnificent monster of the deep learning world, demanding attention with its capacity to digest data sequences with unmatched efficacy. It has a towering self-attention mechanism and powerful feedforward layers. People are in awe of its skills as a result of its skill in multimodal activities, language interpretation, and picture processing. Certainly, artificial intelligence (AI) continues to advance more quickly. Therefore, the preparation, procedure, preservation, and commercialization of the energy infrastructure are hotspots for research approaches based on data-driven technologies AI. In several areas of picture analysis and evaluating, including self-driving automobiles, it performs remarkably well. A brief overview of the transformer neural network and the specified input define are presented in this research. Next, a transformer neural network design accompanying model for tasks involving natural language processing is outlined. Following that, papers relating to distinct transformer neural network types together with distinct techniques and approaches have been addressed here through multiple points of view of different scenarios. This study additionally outlines many transformer neural network applications. Finally, various artificial intelligence-based strategies for transformer design optimising and prospective directions are highlighted.