Lightweight Decoding Strategies for Increasing Specificity
Katy Ilonka Gero, Chris Kedzie, Savvas Petridis, Lydia B. Chilton · arXiv (Cornell University) · 2021
Language models are known to produce vague and generic outputs. We propose two unsupervised decoding strategies based on either word-frequency or point-wise mutual information to increase the specificity of any model that outputs a probability distribution over its vocabulary at generation time. We test the strategies in a prompt completion task; with human evaluations, we find that both strategies increase the specificity of outputs with only modest decreases in sensibility. We also briefly present a summarization use case, where these strategies can produce more specific summaries.