SemEval-2023 Task 8: Causal Medical Claim Identification and Related PIO Frame Extraction from Social Media Posts
Vivek Khetan, Somin Wadhwa, Byron Wallace, Silvio Amir · 2023
Identification of medical claims from usergenerated text data is an onerous but essential step for various tasks including content moderation, and hypothesis generation.SemEval-2023 Task 8 is an effort towards building those capabilities and motivating further research in this direction.This paper summarizes the details and results of shared task 8 at SemEval-2023 which involved identifying causal medical claims and extracting related Populations, Interventions, and Outcomes ("PIO") frames from social media (Reddit) text. 1 This shared task comprised two subtasks: (1) Causal claim identification; and (2) PIO frame extraction.In total, seven teams participated in the task.Of the seven, six provided system descriptions which we summarize here.For the first subtask, the best approach yielded a macro-averaged F-1 score of 78.40, and for the second subtask, the best approach achieved token-level F-1 scores of 40.55 for Populations, 49.71 for Interventions, and 30.08 for Outcome frames.