Productionizing PILAR as a Logstash Plugin
Aaron Abraham, Yash Dani, Kevin Zhang · 2024
Unlike industry log parsing solutions, most academia log parsers can handle changing log events. However, such log parsers are often not engineered to work at the scale of production software. In this paper, we re-architect PILAR, a highly-accurate research log parser, into a multithreaded Ruby plugin built to integrate with Logstash, an industry log management software. Our approach maintains PILAR's high accuracy while significantly improving scalability and efficiency, processing millions of log events per minute.