Characterization of Emerging AI Workloads: Neural Logic Machines and Graph Convolutional Networks
Cory Davis, Patrick Stockton, Eugene B John, Zachary Susskind, Lizy K. John · 2022
The present renaissance of artificial intelligence has created new domains of AI models. Some of these domains include Neuro-Symbolic AI (NSAI) and Graph Neural Networks (GNN). NSAI and GNN models have already demonstrated the capability to outperform state-of-the-art deep learning models in domains such as image and video reasoning, and network classification respectively. They have also been shown to obtain high accuracy with significantly less training data than traditional deep neural network models. Due to the recency of the fields' emergence and relative sparsity of published results, the performance characteristics of these models are not well understood. In this paper, we describe and analyze two recent models in these domains. We find that the symbolic model has less potential parallelism than traditional neural models due to complex control flow and low-operational-intensity operations and high cost of data movement. Additionally in the GNN model, we find an abundance of sparse matrix multiplication. Dense MM has a high potential for parallelism through usage of tensor cores, meanwhile new techniques for increasing parallelism in sparse matrix multiplication will be of extreme importance for GNN models.