An Investigation of Reference Question Difficulty over Time
Kathryn J. Oberg · Carolina Digital Repository (University of North Carolina at Chapel Hill) · 2019
Over the past few decades, reference work has altered as a result of advances in information technology. The purpose of this study is to investigate how questions posed at the Internet Public Library (IPL) virtual reference desk changed over time in difficulty, a concept challenging to define and to measure. Evaluation of reference work has taken a great variety of forms, but no consistent measure examining reference question difficulty has dominated. Few studies have examined reference question difficulty, and those looking at difficulty over time are even rarer. A modified content analysis was used to analyze questions posed to the IPL from 1996 to 2008. An intercoder agreement measure was completed to test reliability. No significant change was found in difficulty of IPL questions over time but satisfactory intercoder reliability was found for the difficulty rating approach. This study supports use of difficulty rating of questions and suggests future research.