Techniques for Gigabyte-Scale N-gram Based Information Retrieval on Personal Computers
Ethan L. Miller, Dan Shen, Junli Liu, Charles Nicholas, Ting Chen · 1999
In this paper, we discuss the implementation techniques that allowed us to use n-gram based retrieval methods on a gigabyte corpus on commodity personal computer hardware. While such techniques have been used before in word-based systems, n-gram systems have different challenges primarily caused by the significantly larger number of unique terms in the corpus. Our work