Insight generation and complex datasets analysis with LLMs in AdTech
Artem Moskovets, Manex Serras · 2024
This study pioneers the application of large language models (LLMs) to the practical task of automating insight generation in the advertising technology (AdTech) sector, a traditionally labor-intensive area demanding substantial human effort. We critically assess the capabilities of GPT-4 and GPT-4o through a replicated analysis of real-world advertising campaign data, containing contextual signals and key performance indicators (KPIs) like impressions, clicks, and engagement rates. Our findings reveal that LLMs have the potential to significantly increase the efficiency of data analysis, succeed at processing complex datasets, and generate actionable insights aligned with the business context. While GPT-4 excels in criteria such as alignment, relevance, and fluency, deploying these models presents challenges, including stochastic behavior and occasional errors in factuality. By describing the original methodology and pinpointing areas for future improvement, this research not only confirms the transformative potential of LLMs in AdTech but also serves as a foundational blueprint for automating business-related data analysis processes.