Duplicate document detection
Larry Spitz · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 1997
In document image filing applications it is important to be able to recognize whether a particular document has already been entered into the system either as an individual document or as an inclusion in another document.Document images could be matched on the basis of layout or contents.However, matching of layout may not be effective when style is strictly controlled. We develop a document 'handle' which is stored along with the document image. The handle is simply a character shape coded representation of the image after the figures and tables have ben removed. Character shape coding is a method of identifying individual character images as members of one of a small number of classes. This process is computationally inexpensive and tolerant of differing generations of photocopying, skew and scanner characteristics. When a new document is entered into the system, its handle is computed and compared against al of the extant handles using a normalized Levenshtein metric. We demonstrate the ability to detect duplicate documents comprising single and multiple pages.