Comparison of Large Language And Vision Models on Representative Downstream Tasks
Huitong Chen · 2023
The success of large model pre-training in the fields of natural language processing and computer vision has been remarkable. By pre-training on large-scale data, these models can learn richer semantic and visual representations to achieve better performance on various tasks. performance. In order to enable more researchers and practitioners to fully understand the advantages and applicable fields of different models, this paper summarizes the current large model pre-training methods, including Vision Transformer (ViT), Pre-trained Transformer (GPT), Text-to-Text Transfer Transformer (T5), Bidirectional Encoder Representations from Transformers (BERT) and Contrastive Language-Image Pretraining (CLIP) are all included, and discusses their principles and advantages are presented. At the same time, compare their results are compared on different datasets, and compare the results of combining them with different other techniques. The final research results show that ViT has advantages over other models in image classification, GPT performs best in generative tasks compared to other models, and T5 has achieved significant results in various text tasks compared to other models, while BERT performs best in various discriminative tasks compared to other models. CLIP is suitable for image and text-related tasks, including image classification, text retrieval, visual question answering, etc. Finally, applications of various models are summarized, providing direction for researchers and practitioners.