Multimodal Approaches for Detecting Mimicry in medicalvideo consultations
Kaihang Wu · The Sydney eScholarship Repository (The University of Sydney) · 2019
Excellent nonverbal communications between doctors and patients are essential for having an effective medical consultation. Current studies usually work on face-to-face doctor-patient communications and manually measure the nonverbal behaviors for doctors or students; however, the main challenge of those methods is to automatize the detection method. In the past decade, new technologies, such as video conferencing, transformed medical communication. Therefore, it is better to do nonverbal behavior analyses via computer vision technologies, which can benefit the video-conferencing medical consultations. This thesis presents a system consisting of detectors to recognize medical student’s nonverbal behaviors automatically. The system implemented the new technologies, including computer vision (CV) and affective computing, to enable automatic detections for video-conferencing communication between medical students (from UNSW) and volunteers who acted as Simulated Patients (SP). The results of the detected medical students’ behaviors have been analyzed and showed that nonverbal behavior mimicry has an impact on their communication performance. In addition, how students’ nonverbal behaviors influence the quality of the teleconsultations has been studied. Communication skills can be vital to many professions, not just medical education. The potential application of nonverbal behavior mimicry detector and the findings of this thesis are not limited. Many professionals, such as employee-employer communication, could be benefited from using detectors for communication skills training.