Probing Representations for Document-level Event Extraction
Barry Wang, Xinya Du, Claire Cardie · 2023
The probing classifiers framework has been employed for interpreting deep neural network models for a variety of natural language processing (NLP) applications.Studies, however, have largely focused on sentencelevel NLP tasks.This work is the first to apply the probing paradigm to representations learned for document-level information extraction (IE).We designed eight embedding probes to analyze surface, semantic, and event-understanding capabilities relevant to document-level event extraction.We apply them to the representations acquired by learning models from three different LLM-based document-level IE approaches on a standard dataset.We found that trained encoders from these models yield embeddings that can modestly improve argument detections and labeling but only slightly enhance event-level tasks, albeit trade-offs in information helpful for coherence and event-type prediction.We further found that encoder models struggle with document length and cross-sentence discourse.