Variability of interobserver interpretation of selected helminth ova in the development of a training image set
Rupert Stephen Charles Chua, Kiersten A Henderson, Lorenzo Maria C. de Guzman, Vicki Foss, Nathaniel Schub, Cameron Bell, John Robert C. Medina, Taggart G. Siao, Myra S. Mistica, Maria Luz B. Belleza, Marie Cris R Modequillo, Nadine Joyce C Torres, Vicente Y. Belizario · International Health · 2024
BACKGROUND: Diagnosis of soil-transmitted helminthiasis and schistosomiasis for surveillance relies on microscopic detection of ova in Kato-Katz (KK) prepared slides. Artificial intelligence (AI)-based platforms for parasitic eggs may be developed using a robust image set with defined labels by reference microscopists. This study aimed to determine interobserver variability among reference microscopists in identifying parasite ova. METHODS: Images of parasite ova taken from KK prepared slides were labelled according to species by two reference microscopists (M1 and M2). A third reference microscopist (M3) labelled images when the first two did not agree. Frequency, percent agreement, κ statistics and variability score (VS) were generated for analysis. RESULTS: M1 and M2 agreed on 89.24% of the labelled images (κ=0.86, p<0.001). M3 had agreement with M1 and M2 (κ=0.30, p<0.001 and κ=0.28, p<0.001), resolving 89.29% of disagreement between them. The labelling of Schistosoma japonicum had the highest VS (κ=0.487, p=0.101) among the targeted ova. Reference microscopists were able to reliably reach consensus in 99.0% of the dataset. CONCLUSIONS: Training AI using this image set may provide more objective and reliable readings compared with that of reference microscopists.