The Gold Standard in Corpus Annotation.
Lars Wißler, Mohammed Almashraee, Dagmar Monett, Adrian Paschke · 2014
Trustworthy corpora are necessary for training and meaningful evaluation of algorithms which use annotations. These standard collections are called Gold Standard Corpora (GSC). However the construction of GSC is a laborious and time-consuming process and size, quality and most of all availability of task-specific GSC directly influence the development of machine learning based natural language processing algorithms. This paper provides an introduction to gold standard corpus construction in the context of natural language processing and gives an overview of alternative approaches.