Development of a Bayesian Thurstonian Model for Analysing Ranking Data From Live Postdoc Recruitment
Noam Tal-Perry, Lara Abel, Mollie Etheridge, Jessica Hampton, Becky Ioppolo, Adrian Gerard Barnett, Timothy R. Johnson, Steven Wooding · 2025
Various academic stakeholders are currently exploring narrative curriculum vitae (CV) in funding and recruitment to address research culture concerns. In a recent pilot using a randomised controlled trial (RCT), we asked applicants to submit both a standard and a narrative CV when applying to postdoctoral positions at the University of Cambridge, with panel members randomly assigned one format for initial evaluation and ranking. Here, we use this ranking data to develop a multi-layered generative model that simulates different recruitment scenarios by systematically varying the model’s parameters. We analysed the results using a Bayesian Thurstonian model for the analysis of ranked data from multiple raters to test whether parameters were correctly recovered and how the change in parameters affected the uncertainty around model estimates. The results from this process were used to inform the design of the main phase implementation of the pilot study design in a larger sample.