Weakly- and Semi-supervised Evidence Extraction
Danish Pruthi, Bhuwan Dhingra, Graham Neubig, Zachary C. Lipton · 2020
For many prediction tasks, stakeholders desire not only predictions but also supporting evidence that a human can use to verify its correctness.However, in practice, evidence annotations may only be available for a minority of training examples (if available at all).In this paper, we propose new methods to combine few evidence annotations (strong semisupervision) with abundant document-level labels (weak supervision) for the task of evidence extraction.Evaluating on two classification tasks that feature evidence annotations, we find that our methods outperform baselines adapted from the interpretability literature to our task.Our approach yields gains with as few as hundred evidence annotations. 1 Explanations Evidence Objective Elucidate "the reasons behind predictions".Enable users to quickly verify the predictions. EvaluationExplanations are specific to the model.No ground truth explanations to compare against.Evidence is a characteristic of the task.Can be compared against human-labeled evidence. ExampleA horror movie that lacks cohesion.A horror movie that lacks cohesion.