An Adaptive Elastic Multi-model Big Data Analysis and Information Extraction System
Qiang Yin, Jianhua Wang, Sheng Du, Jianquan Leng, Jintao Li, Yinhao Hong, Feng Zhang, Yunpeng Chai, Xiao Zhang, Xiaonan Zhao, Mengyu Li, Song Xiao, Wei Lu · Data Science and Engineering · 2022
Abstract With the diverse applications to industry and domain-specific context, multi-source information extraction on semi-structured and unstructured data, as well as across data models, is becoming more common. However, multi-model information extraction often requires the deployment of multiple data model management, storage, and analysis subsystems on the cloud, many subsystems are not high-resource utilization at the same time, and the resource waste phenomenon is often serious. Therefore, an adaptive scalable multi-model big data analysis and information extraction system is designed and implemented in this paper, which can support data maintenance and cross-model query of relational, graph, document, key and other data models, and can provide efficient cross-model information extraction. On this basis, we can achieve the system resource allocation on demand and fast scaling mechanism, according to the real-time requirements of multi-model big data analysis, and dynamic adjustment of each subsystem resource allocation. Therefore, our solution not only guarantees multi-model query and information extraction performance and quality of service, but also significantly reduces the total consumption of system resources and cost.