Tabular Data Classification via an Improved Gated Transformer
Lu Huang, Anjie Peng · 2024
Tabular data classification is a key task in the field of data analysis, such as finance, healthcare, and business intelligence. Although traditional Transformer models perform well at working with sequential data, they face challenges when working with structured tabular data. In this paper, an improved gating mechanism is proposed to be integrated into the Transformer architecture to enhance the model's ability to capture the features of tabular data, aiming at improving the classification performance for structured tabular data. To this end, the tabular data is first unstructured by some preprocessing operations, making the structured tabular data be same as the sequential data for the convenience of using powerful natural language processing tools. After the preprocessing step, the unstructured tabular data is converted into an embedding vector to capture the initial feature. Then, these embedding vectors are further encoded by the proposed improved gated Transformer architecture to capture the intrinsic characteristic of the data. We propose a new gating mechanism to dynamically adjust the information flow, which improving the sensitivity of the Transformer model to key features. Finally, the feature is fed into a linear classifier for classification. Extensive experiments on two public tabular datasets verify the effectiveness of the proposed method for the tabular data classification. In addition, the proposed method achieved a higher F1-score than existing methods.