Comparing Apples and Oranges: Human and Computer Clustered Affinity Diagrams Under the Microscope
Parzival Borlinghaus, Stephan L. Huber · 2021
Affinity diagramming is a crucial yet time-consuming part of user research in human-centered design. In short, building affinity diagrams involves the hierarchical bottom-up clustering of user statements and observations, which later allow to derive insights and inspire design ideas. To support designers in this process, as a first contribution, we explored seven text-mining models for pre-clustering affinity notes and suggest fastText as most appropriate. Since affinity diagrams are not deterministic, there is no established measure to assess their quality. Our second contribution is, therefore, a thorough examination of the potential of fastText-clusters for design teams regarding technical, psychological and performance-related measures. Compared to reference ‘human built’ affinity diagrams, the fastText-clusters resulted in an overlap index of M = .694 (SD = .034). Surprisingly, a study with four design teams clustering small sets (112 notes) of pre-clustered or randomized affinity notes indicated an increased discussion overhead caused by algorithmic support that led to a decrease in both, efficiency and quality. As a third contribution, we report qualitative data from the instances, where algorithmic support failed designers’ expectations. We conclude that more research on the appropriate time and manner of pre-clustered data presentation is required to harness the full potential of algorithmic support while preserving the spirit of affinity diagramming.