Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revision
Qian Ruan, Ilia Kuznetsov, Iryna Gurevych · 2024
Collaborative review and revision of textual documents is the core of knowledge work and a promising target for empirical analysis and NLP assistance.Yet, a holistic framework that would allow modeling complex relationships between document revisions, reviews and author responses is lacking.To address this gap, we introduce Re3, a framework for joint analysis of collaborative document revision.We instantiate this framework in the scholarly domain, and present Re3-Sci, a large corpus of aligned scientific paper revisions manually labeled according to their action and intent, and supplemented with the respective peer reviews and human-written edit summaries.We use the new data to provide first empirical insights into collaborative document revision in the academic domain, and to assess the capabilities of state-of-the-art LLMs at automating edit analysis and facilitating text-based collaboration.We make our annotation environment and protocols, the resulting data and experimental code publicly available.1