Combining Minimal Surface Model with DGCNN for Natural Language Understanding
Mingkai Zha, Gongqi Lin, Hongguang Fu, Yunhao Liu, Lei Huang · 2021 2nd International Conference on Electronics, Communications and Information Technology (CECIT) · 2021
Fine-tuning large pre-trained models is an adequate transfer mechanism in Natural Language Processing (NLP). However, it is challenging to serve in resource-restricted devices due to being computationally expensive. To reduce the trainable parameters of models while maintaining accuracy, we propose a novel geometry deep learning approach named Minimal Surface Model on DGCNN (MSM-D). In MSM-D, the minimal surface model is introduced as an adapter layer to convert the word vector into high-dimensional manifold data, an input to the point cloud model. MSM - D is empirically adequate and achieves close or even better performance of BERT-base on GLUE benchmark while being 5.7x smaller on inference.