Chuweb21D: A Deduped English Document Collection for Web Search Tasks
Zhumin Chu, Tetsuya Sakai, Qingyao Ai, Yiqun Liu · 2023
As a traditional information retrieval task, ad hoc web search has long been an important part of IR research and evaluation tracks (e.g. TREC, NTCIR and CLEF). A crawled, large-scale web document collection is a central component to offline web search evaluation. Although there already exist several English document collections, such as the ClueWeb series, GOV2 and c4, a collection that satisfies properties of both strong timeliness and raw HTML formatting is still relatively scarce.