Finding Appropriate Lexical Diversity Measurements for Small-Size Corpus
Woon Ho Choi · Applied Mechanics and Materials · 2011
In the present investigation four kinds of lexical diversity measurement have been applied to the sets of word chunks with monotone increasing size. The computational experiment with corpus processing and statistical test has been conducted to find out the most effective lexical diversity measurement in evaluating a small-sized corpus of 350~550 words, and the result shows that D-estimate is the most appropriate among the four lexical diversity measurements which are considered in this research. Also D-estimate shows more stable results than other measurements when the number of words varies between texts.