Improving Performance Through Object Lifetime Profiling: the DataFrame Case
Sebastian Jordan Montaño, Nahuel Palumbo, Guillermo Polito, Sté́phane Ducasse, Pablo Tesone · HAL (Le Centre pour la Communication Scientifique Directe) · 2023
Being capable of profiling the object lifetimes of an application gives information that can be used to optimize the GC performance and improve overall execution time. One can pre-tenure objects based on profiler information, tune the GC parameters, or take decisions about pre-allocating bigger memory segments. However, accessing object lifetimes is difficult because it requires monitoring any object GC reclamation. We developed an open-source lifetime profiler. Our current implementation does not require Virtual Machine modification. It is based on ephemerons and method proxies. We profiled DataFrame and we observed a significant number of objects that lived a long time. We used this information to tune the garbage collector parameters and we got up to 6.8 times of performance improvements.