Applying Generative AI to Create SOP, Reducing API Costs Through Prompt Compression and Evaluating LLM Responses with Tonic Validate RAG Metrics
T. Vetriselvi, Mihir Mathur, M. Bhuvaneswari · 2024
Generative Artificial Intelligence (AI) utilizes existing data to create new forms of content, including text, images, and audio. One valuable application of generative AI in an IT Enabled Services (ITES) company is the development of Standard Operating Procedures (SOPs) by processing and consolidating existing SOPs through a Large Language Model. This paper explores how generative AI can generate content within standardized templates, improve the language of SOPs, and make them suitable for specific industries and processes, such as the pharmaceutical procure-to-pay process. It also addresses version control, helps users maintain consistency and compliance, extracts knowledge for onboarding, and provides interactive training on SOPs through Questions and Answers. The proposed workflow employs Azure OpenAI's GPT-3.5 turbo for generating responses, which are evaluated using Tonic Validate Retrieval Augmented Generation (RAG) metrics. Furthermore, the paper introduces Prompt Compression approaches and selects one prompt compression approach to streamline the context retrieved for large language models while preserving the semantic meaning of the outputs. It also details strategies for reducing API call costs by over 10% for prompts of varying token sizes, ensuring high-quality responses from the GPT-3.5 turbo model according to RAG metrics. These criteria of accuracy, precision and cost reduction are considered to recommend the prompt compression approach for Large Language Models used in development of Standard Operating Procedures.