Web Genre Benchmark Under Construction
Marina Santini, Serge Sharoff · LDV-Forum/Journal for language technology and computational linguistics · 2009
Web Genre Benchmark Under ConstructionThe project presented in this article focuses on the creation of web genre benchmarks (a.k.a.web genre reference corpora or web genre test collections), i.e. newly conceived test collections against which it will be possible to judge the performance of future genre-enabled web applications.The creation of web genre benchmarks is of key importance for the next generation of web applications because, at present, it is impossible to evaluate existing and in-progress genre-enabled prototypes.We suggest focusing on the following key points: ) propose a characterisation of genre suitable for digital environments and empirical approaches shared by a number of genre experts working in automatic genre identification; ) define the criteria for the construction of web genre benchmarks and draw up annotation guidelines; ) create web genre benchmarks in several languages; ) validate the methodology and evaluate the results.We describe work in progress and our plans for future development.Since it is sometimes difficult to anticipate the difficulties that will arise when developing a large resource, we present our ideas, our current views on genre issues and our first results with the aim of stimulating a proactive discussion, so that the stakeholders, i.e. researchers who will ultimately benefit from the resource, can contribute to its design. The Concept of GenreThe concept of genre is hard to agree upon.Many interpretations have been proposed since Aristotle's Poetics without reaching any definite conclusions about the inventory or even principles for classifying documents into genres.Some studies put the number of genres to , (Görlach, ) or even , (Adamzik, ).Additionally, the lack of an agreed definition of what genre is causes the problem of the loose boundaries between the term 'genre' with other neighbouring terms, such as 'register', 'domain', 'topic', and 'style'.The inventory of genres can be based on linguistic theories or 'folksonomies', i.e. labels used by users (Rosso and Haas, ming).For instance, users are confident with a term like novel, whereas linguistic researchers may prefer functional terms, like recreation to indicate a wider range of texts aimed at recreational reading.Recently, definitions of genre have been adapted to the new digital environments, e.g., (Yates and Orlikowski, ; Erickson, ; Toms and Campbell, ; Beghtol, ; Heyd, ; Bateman, ).Undoubtedly, the situation on the web is more difficult than in the offline world, because the web is new, genres are fluid, web documents are very often characterised by a high level of hybridism, by the fragmentation of textuality across several documents, by the impact of technical features such as JLCL 2009 -Volume 24 (1) -129-145