Transformative Neural Networks for Technical Text Generation: Context Length Dependence Analysis
Ihor Mykhailichenko, Heorhii Ivashchenko, Тетяна Володимирівна Філімончук, Oleksii Liashenko · 2024
The article proposes a transformer-based neural network trained on a technical book focused on computer architecture, aimed at generating coherent technical text. The research primarily investigates the impact of context length on the quality and coherence of generated text. It explores how variations in the input context length influence the model's ability to generate text that closely resembles the style and content of the training material. Experimental results demonstrate significant variations in output quality based on context length, providing insights into optimizing transformer models for technical text generation. These findings have implications for the development of more efficient and effective automated documentation tools in the field of computer engineering.