Evaluating the frequency threshold for selecting lexical bundles by means of an extension of the Fisher's exact test
Yves Bestgen · Corpora · 2018
This methodological study applies an extension of the Fisher's exact test to sequences longer than two words in a bid to evaluate whether the frequency thresholds that are conventionally used for identifying lexical bundles in corpora are high enough to ensure that the selected sequences are unlikely to result from chance. Sequences of four, three and two words were analysed in corpora, the sizes of which ranged from 50,000 to 4,200,000 words. Results suggested that the usual frequency cut-offs were appropriate for four-word sequences and that, as expected, it was questionable to extract two-word sequences on the sole basis of a frequency criterion. For three-word sequences, greater cut-offs than forty times per million words should be favoured for corpora with a size of 250,000 words or fewer. This study also highlights the effect of corpus size on the efficiency of the frequency thresholds when they are expressed in normalised frequency.