Cross-Modal Multi-Source Public Data Fusion and Retrieval using Knowledge Distillation Method
Zichen Huang, Shaobo Li, Jing Yang, Xingxing Zhang, Jinhu Zhang · 2023
In the burgeoning field of Vision-Language Pretrained Models (VLPs), there is a notable gap in the application and development of these technologies in the Chinese public healthcare sector. Despite the promising advancements of VLPs in English-centric settings, their adaptation for multimodal data fusion in Chinese remains inadequately explored. This paper addresses this lacuna by introducing a specialized Chinese cross-modal data fusion retrieval model, grounded in the framework of knowledge distillation. Utilizing a novel, multi-modal healthcare dataset tailored for the Chinese context, the model is fine-tuned through knowledge distillation techniques to optimize data fusion strategies. The study identifies critical challenges—such as model scarcity, data bottleneck, and cultural-linguistic barriers—hampering the efficient and secure transmission of multimodal data in Chinese healthcare applications. By leveraging knowledge distillation, our approach demonstrates substantial improvements in data retrieval efficiency and security, thus laying a foundational groundwork for future advancements in public data management within the Chinese context.