On the Gap between Adoption and Understanding in NLP
Federico Bianchi, Dirk Hovy · 2021
There are some issues with current research trends in NLP that can hamper the free development of scientific research.We identify five of particular concern: 1) the early adoption of methods without sufficient understanding or analysis; 2) the preference for computational methods regardless of risks associated with their limitations; 3) the resulting bias in the papers we publish; 4) the impossibility of re-running some experiments due to their cost; 5) the dangers of unexplainable methods.If these issues are not addressed, we risk a loss of reproducibility, reputability, and subsequently public trust in our field.In this position paper, we outline each of these points and suggest ways forward.