Understanding human quality judgment in assessing online forum contents for thread retrieval purpose
Zuriati Ismail, Naomie Binti Salim, Sharin Hazlin Huspi · AIP conference proceedings · 2017
Compared to traditional materials or journals, user-generated contents are not peer-reviewed. Lack of quality control and the explosive growth of web contents make the task of finding quality information on the web especially critical. The existence of new facilities for producing web contents such as forum makes this issue more significant. This study focuses on online forums threads or discussion, where the forums contain valuable human-generated information in a form of discussions. Due to the unique structure of the online forum pages, special techniques are required to organize and search for information in these forums. Quality biased retrieval is a retrieval approach that search for relevant document and prioritized higher quality documents. Despite major concern of quality content and recent development of quality biased retrieval, there is an urgent need to understand how quality content is being judged, for retrieval and performance evaluation purposes. Furthermore, even though there are various studies on the quality of information, there is no standard framework that has been established. The primary aim of this paper is to contribute to the understanding of human quality judgment in assessing online forum contents. The foundation of this study is to compare and evaluate different frameworks (for quality biased retrieval and information quality). This led to the finding that many quality dimensions are redundant and some dimensions are understood differently between different studies. We conducted a survey on crowdsourcing community to measure the importance of each quality dimensions found in various frameworks. Accuracy and ease of understanding are among top important dimensions while threads popularity and contents manipulability are among least important dimensions. This finding is beneficial in evaluating contents of online forum.