Towards End-to-End Optimization of LLM-based Applications with Ayo
Xin Tan, Yimin Jiang, Yitao Yang, Hong Li Xu · 2025
Large language model (LLM)-based applications consist of both LLM and non-LLM components, each contributing to the end-to-end latency. Despite great efforts to optimize LLM inference, end-to-end workflow optimization has been overlooked. Existing frameworks employ coarse-grained orchestration with task modules, which confines optimizations to within each module and yields suboptimal scheduling decisions.