A benchmark of online COVID-19 symptom checkers
Nicolas Munsch, Alistair G. Martin, Stefanie Gruarin, Jama Nateqi, Isselmou Abdarahmane, Rafael Weingartner-Ortner, Bernhard Knapp · medRxiv · 2020
Abstract Background A large number of online COVID-19 symptom checkers and chatbots have been developed but anecdotal evidence suggests that their conclusions are highly variable. To our knowledge, no study has evaluated the accuracy of COVID-19 symptom checkers in a statistically rigorous manner. Methods In this paper, we evaluate 10 different COVID-19 symptom checkers screening 50 COVID-19 case reports alongside 410 non-COVID-19 control cases. Results We find that the number of correctly assessed cases varies considerably between different symptom checkers, with Symptoma (F 1 =0.92, MCC=0.85) showing the overall best performance followed by Infermedica (F 1 =0.80, MCC=0.61).