Testing diachronic measures of productivity using the Zipf-Mandelbrot law
Quentin Feltgen · Amsterdam studies in the theory and history of linguistic science. Series 4, Current issues in linguistic theory · 2025
Abstract The productivity of patterns (be it morphological or syntactic) has long been studied in diachrony. However, the study of the relationship between productivity and token frequency has been hindered by the non-trivial dependency of productivity on the sample size on which it is computed. Several solutions have been offered in the literature but a systematic evaluation of them all has not been performed to this date. In this paper, we rely on the Zipf-Mandelbrot distribution to generate synthetic data and perform a statistical evaluation of these methods. Our result is that many of these methods are successful at providing a measure independent of the sample size, and that they are closely correlated.