ConTraPh: Contrastive Learning for Parallelization and Performance Optimization
Quazi Ishtiaque Mahmud, Ali TehraniJamsaz, Nesreen K. Ahmed, Theodore L. Willke, Ali Jannesari · 2025
With the advancement of HPC platforms, the demand for high-performing applications continues to grow.One effective way to enhance program performance is through parallelization.However, fully leveraging the powerful hardware of HPC platforms poses significant challenges.Even experienced developers must carefully consider factors such as runtime, memory usage, and thread-scheduling overhead.Additionally, achieving successful parallelization often requires running applications to determine the optimal configurations.In this paper, we propose ConTraPh, a framework that integrates Contrastive Learning with Transformers and Graph Neural Networks to capture the inherent parallel characteristics of source programs through a multi-view program representation, utilizing both source code and compiler intermediate representations.This contrastive learning framework allows the model to effectively learn correct parallel configurations from positive samples while avoiding incorrect ones through negative samples.We evaluate Con-TraPh on six downstream tasks involving three different parallel programming models OpenMP, OpenCL and, Ope-nACC that include OpenMP clause prediction, performant reduction style detection, performant scheduling type detection, CPU/GPU parallelism prediction, Heterogeneous Device Mapping for OpenCL code, and OpenACC clause prediction.ConTraPh outperforms state-of-the-art models in these tasks, achieving accuracy improvements of up to 8%, 10%, 7%, 4%, 2%, and 9%, respectively.ConTraPh achieves speedups as high as 13x, 18x, 14x, and 4.4x on the reduction