Estimating Vocabulary Size
Alvar Ellegård · WORD · 1960
Estimating Vocabulary SizeHow many words are there in a language like English?What is the normal vocabulary of an educated adult native speaker?How many words should a student know at various stages of learning a foreign language?What do we mean.by saying that a writer has a rich vocabulary?How many words does an ordinary novel contain?How often should we expect a dictionary containing 10,000 (or 50,000, or 100,000) entries to fail to give us any lead in the course of reading an ordinary book of 300 pages?These are some of the questions concerning vocabulary size which are of interest to linguists, psychologists, teachers, lexicographers (including machine translators), and perhaps even to ordinary people.The present paper is an attempt to provide some of the basic facts and arguments which are needed to answer them.l 1.The first problem to be treated is this: Given an average text consisting of W running words, how many different words V will it contain, and how many times will each word in that vocabulary occur ?2It might seem that a proper answer to that question would involve a tremendous amount of sheer mechanical counting of a large number of text samples of different sizes.For instance, we might select, say, ten samples of I 0,000 words, another ten of 20,000, and so on.On the basis of such counts, a diagram could be drawn up,.from which the average number of different words to be expected in a text mass of any size could be read off.Some data which might be included in such a diagram are already in existence: a few will be discussed below.But they are not nearly sufficient, and to obtain what is needed would be a formidable and costly undertaking, even with the aid of electronic data processing machines) It is 1 Some of these problems have been treated in G. U. Yule's classical The Statistical Study of Literary Vocabulary, Cambridge, 1944: and more recently in Pierre Guiraud, Les Caracteres Statistiques du Vocabulaire, Paris, 1954. 2 The symbols used are Yule's.W stands for "words," V for "vocabulary."3 One possible procedure is described in detail in A. 0. Booth, L.