Developing a Benchmark for Reducing Data Bias in Authorship Attribution
Benjamin Murauer, Günther Specht · 2021
Authorship attribution is the task of assigning an unknown document to an author from a set of candidates.In the past, studies in this field use various evaluation datasets to demonstrate the effectiveness of preprocessing steps, features, and models.However, only a small fraction of works use more than one dataset to prove claims.In this paper, we present a collection of highly diverse authorship attribution datasets, which better generalizes evaluation results from authorship attribution research.Furthermore, we implement a wide variety of previously used machine learning models and show that many approaches show vastly different performances when applied to different datasets.We include pre-trained language models, for the first time testing them in this field in a systematic way.Finally, we propose a set of aggregated scores to evaluate different aspects of the dataset collection.