Spectrum-BERT: Pretraining of Deep Bidirectional Transformers for Spectral Classification of Chinese Liquors
Yansong Wang, Yundong Sun, Yansheng Fu, Dongjie Zhu, Zhaoshuo Tian · IEEE Transactions on Instrumentation and Measurement · 2024
Counterfeit Chinese liquor incidents in China have significantly disrupted market order and jeopardized the health of consumers. Currently, deep learning-based spectral detection techniques are extensively leveraged in non-invasive food inspection. Excessive reliance on labels severely limits its application in real scenarios. To make better use of limited samples, we are the first to use the “unsupervised pre-training & supervised fine-tuning” paradigm in combining the Transformer architecture for feature extraction and classification of the Chinese liquor spectrum, and propose Spectrum-BERT, which represents Bidirectional Encoder Representations from Transformers for Spectrum. Specifically, we creatively propose spectral curve partitioning and 1-D convolutional layer mapping to maintain the model’s sensitivity to characteristic peak locations and local information of spectral curves. Moreover, the paradigm of “unsupervised pre-training & supervised fine-tuning” addresses the limitation of label deficiency, thus improving the model’s applicability. Finally, we have conducted extensive experiments on the real liquor spectral dataset. Comparative experiments demonstrate that Spectrum-BERT outperforms all baselines on all metrics using only 70% supervised signal. The limit experimental results show that Spectrum-BERT can still maintain its lead using only the 10% supervised signal. Thanks to the more efficient model architecture, Spectrum-BERT’s model parameters and FLOPs are only 1/568 and 1/322 of those of baselines.