Design of an Active Learning System with Human Correction for Content Analysis
Nancy J. McCracken, Jasy Suet Yan Liew, Kevin Crowston · 2014
Our research investigation focuses on the role of humans in supplying corrected examples in active learning cycles, an important aspect of deploying active learning in practice. In this paper, we dis-cuss sampling strategies and sampling sizes in set-ting up an active learning system for human ex-periments in the task of content analysis, which involves labeling concepts in large volumes of text. The cost of conducting comprehensive hu-man subject studies to experimentally determine the effects of sampling sizes and sampling sizes is high. To reduce those costs, we first applied an active learning simulation approach to test the ef-fect of different sampling strategies and sampling sizes on machine learning (ML) performance in order to select a smaller set of parameters to be evaluated in human subject studies. 1