Beyond Aesthetics: Evaluating Response Widgets for Reliability & Construct Validity of Scale Questionnaires
Habiba Farzand, David Al Baiaty Suarez, Thomas Alexander Goodge, Shaun Alexander Macdonald, Karola Marky, Mohamed A. Khamis, Paul A. Cairns · 2024
Scale questionnaires are psychometric tools that capture perspectives and experiences. Consequently, these tools need to be reliable and valid. In this paper, we investigate the impact of response widgets - the UI elements that allow users to answer scale items - on the overall scale reliability and construct validity of three varied length scale questionnaires in a user study (N=30). Our results reveal that optimum reliability was achieved using radio buttons and dropdowns in all varied-length questionnaires. Further, valid results were produced utilising the slider and dropdown. No significant differences were found in time consumption, but click count was significantly higher with dropdown. Radio buttons scored lower in format satisfaction than others, and dropdown was the least effective in ease of selection and quick completion. In light of these results, we conclude that response widgets are more than just aesthetics and should be selected as per the researcher’s aims.