A COMPARISON OF CORPUS-BASED TECHNIQUES FOR RESTORING ACCENTS
In Spanish, French Text · 1999
This chapter will explore and compare three corpus-based techniques for lexical ambiguity resolution, focusing on the problem of restoring missing accents to Spanish and French text. Many of the ambiguities created by missing accents are dif ferences in part of speech: hence one of the methods considered is an N-gram tagger using Viterbi decoding, such as is found in stochastic part-of-speech taggers. A second technique, Bayesian classification, has been successfully applied to word-sense disam biguation and is well suited for some of the semantic ambiguities which arise from missing accents. The third approach, based on decision lists, combines the strengths of the two other methods, incorporating both local syntactic patterns and more distant collocational evidence, and outperforms them both. The problem of accent restoration is particularly well suited for demonstrating and testing the capabilities of the given algorithms because it requires the resolution of both semantic and syntactic ambi guity, and offers an objective ground truth for automatic evaluation. The problem is also a practical one with immediate application. 1. PROBLEM DESCRIPTION Accent restoration is closely related to several lexical disambiguation problems. It involves aspects of both word-sense disambiguation and part-of-speech tagging. While not as widely cited as these other tasks, it nonetheless offers considerable benefits as a case study, and is particu larly useful for evaluating and comparing the disambiguation algorithms considered here. Specifically: It requires the resolution of both syntactic and semantic ambigui ties, and is representative of many of the issues that arise in several important types of lexical ambiguity resolution. Unlike many ambiguity resolution tasks which depend on human annotations or judgements for evaluation, this problem supports fully automatic evaluation and an innate, plentiful and objective ground truth: text with accents may be artificially stripped, leaving accent less text for testing purposes with a known gold standard for evaluation.