To Annotate More Accurately or to Annotate More
Dmitriy Dligach, Rodney D. Nielsen, Martha Stone Palmer · University of North Texas Digital Library (University of North Texas) · 2010
The common accepted wisdom is that blind double annotation followed by adjudication of disagreements is necessary to create training and test corpora that result in the best possible performance. We provide evidence that this is unlikely to be the case. Rather, the greatest value for your annotation dollar lies in single annotating more data. 1