The Proposal Auto-Categorizer and Manager for Time Allocation Review at the Space Telescope Science Institute
Louis-Gregory Strolger, Sophia Porter, Jill Lagerstrom, Sarah Weissman, I. Neill Reid, Michael Garcia · The Astronomical Journal · 2017
Abstract The Proposal Auto-Categorizer and Manager (PACMan) tool was written to respond to concerns about subjective flaws and potential biases in some aspects of the proposal review process for time allocation for the Hubble Space Telescope (HST), and to partially alleviate some of the anticipated additional workload from the James Webb Space Telescope (JWST) proposal review. PACMan is essentially a mixed-method Naive Bayesian spam filtering routine, with multiple pools representing scientific categories, that utilizes the Robinson method for combining token (or word) probabilities. PACMan was trained to make similar programmatic decisions in science category sorting, panelist selection, and proposal-to-panelists assignments to those made by individuals and committees in the Science Policies Group (SPG) at the Space Telescope Science Institute. Based on training from the previous cycle’s proposals, at an average of 87%, PACMan made the same science category assignments for proposals in Cycle 24 as the SPG. Tests for similar science categorizations, based on training using proposals from additional cycles, show that this accuracy can be further improved, to the level. This tool will be used to augment or replace key functions in the Time Allocation Committee review processes in future HST and JWST cycles.