A Case Study on Enhancing Inquiry Response in a Non-Life Insurance Company Using Generative AI.
Shojiro Tsutsui, Michihiro Karino, Kenichi Kuroki, Aya Fukumoto, Yusuke Hamano, Kenji Sobata, Temma Saito, Tatsunori Kawamoto, Taku Odashima, Tsuyoshi Kato, Yosuke Motohashi · 2024
In Japan, non-life insurance companies deliver products through agencies. Major insurance companies provide support through phone calls, emails, etc., at locations nationwide to ensure that their tens of thousands of agents can accurately handle customers, taking into account the characteristics and underwriting rules of a wide variety of insurance products. The documents to be referred to cover a vast amount array of complex rules, and as financial products, precise and courteous responses are always needed in accordance with individual cases are vital. In this study, we developed an inquiry response support system using the retrieval-augmented generation (RAG) architecture of large language models (LLMs) with the aim of improving the inquiry response operations of non-life insurance companies. In addition, we conducted evaluation experiments on the optimal combinations of conditions related to response performance, such as the chunk division units of the target manuals for searching and the number of tokens input into the LLM. Our findings showed that the accuracy improved with an appropriate number of input tokens and item-based division units with meaningful content. In the end, the inquiry response system developed with the proposed architecture is in practical use, with 14,000 users utilizing it in their daily operations.