Multimodal aided neurological disease diagnosis with synergy of cloud and client
Junjun FAN, Dongqi Han, Li Yang, Jin Huang, Hongan Wang, Xiaolan Peng, Guozhong Dai, Hui Chen, Feng TIAN · Scientia Sinica Informationis · 2017
The dominating physiological characteristics of neurological diseases are reflected in patients' daily behaviors. The aided diagnosis and early warning of neurological diseases will benefit from obtaining and analyzing related physiological information generated during the interaction process. Traditional systems for detecting neurological conditions only analyze a single interaction modal, which may lose important features contained in other modalities. Based on the above, we propose a multi-modal aided neurological disease diagnosis system with synergy of cloud and client. First, we propose an automatic disease diagnosis method based on multi-modal information; users' physiological information collected from multiple modals is then analyzed and the results are integrated to improve accuracy and robustness. Second, a framework of cloud-client synergy is proposed that stores the user's physiological information from different regions and different times in the cloud, thus reducing the geographical restrictions and time constraints of data collection. Third, based on the powerful computing capacity, the system can make real-time and precise automatic diagnosis by analyzing multi-modal physiological information produced by users during natural interaction. Finally, the effectiveness of this multi-modal method for diagnosing aided neurological diseases based on cloud-client synergy is verified by using a hybrid diagnostic system of voice and pen.