An Exploration of Encoder-Decoder Approaches to Multi-Label Classification for Legal and Biomedical Text
Yova Kementchedjhieva, Ilias Chalkidis · 2023
Standard methods for multi-label text classification largely rely on encoder-only pretrained language models, whereas encoderdecoder models have proven more effective in other classification tasks.In this study, we compare four methods for multi-label classification, two based on an encoder only, and two based on an encoder-decoder.We carry out experiments on four datasets-two in the legal domain and two in the biomedical domain, each with two levels of label granularity-and always depart from the same pre-trained model, T5.Our results show that encoder-decoder methods outperform encoderonly methods, with a growing advantage on more complex datasets and labeling schemes of finer granularity.Using encoder-decoder models in a non-autoregressive fashion, in particular, yields the best performance overall, so we further study this approach through ablations to better understand its strengths.