The Length and Verbal Anchors Do Not Matter: The Influence of Various Likert-Like Response Formats on Scales’ Psychometric Properties
Petra Hubatka, Hynek Cígler, David Elek, Martin Tancoš · 2024
While the Likert scale is the widely used response format to measure personality traits, consensus on how its parameters moderate the performance has not been achieved. We performed two within-subject high-powered experiments, manipulating the extremity of outer (“strongly dis/agree” vs. “dis/agree”) and presence of inner (all points vs. only endpoints labeled) verbal anchors on a 5-point Likert scale (Study 1, N1 = 1044) and a number of options (two, six, and ten) in response format (Study 2, N2 = 846). We used a similar methodology and statistical approach in both studies, focusing mainly on a measurement model and criterion validity. Moreover, we utilized the Height Inventory to compare participants’ scores with the actual height of people and replicated most results using a more typical, standard psychological questionnaire. While the all-labeled, non-extreme, and longer scales have negligibly higher reliability, the criterion validity of observed scores was only (and negligibly) related to the extremity of outer verbal anchors (with higher reliability in the non-extreme variants). Finally, we demonstrated that the measurement model is comparable across all the experimental conditions, leading to the equivalent single latent trait with the same population characteristics and relations to the external criterion. The conclusion is that the performance of the Likert scale is stable across the conditions we manipulated, especially if structural equation modeling is used instead of raw score analysis. Still, we suggest using all-labeled, non-extreme verbal anchors instead. While the two-option Likert scales might have lower reliability, their criterion validity seems to be unimpacted.