Deduplication in large web corpora
Владимир Бенко · Publication Server of the Institute for German Language (Institute for German Language) · 2019
Our paper tries to find answers to some questions related to deduplication process in large-scale web-crawled corpora. An experiment based on eight corpora from the Aranea family is introduced, and first results are presented.