Error Analysis of ToTTo Table-to-Text Neural NLG Models
Barkavi Sundararajan, Somayajulu Gowri Sripada, Ehud Reiter · 2022
We report error analysis of outputs from four Table -to-Text generation models fine-tuned on ToTTo, an open-domain English language dataset.We carried out a manual error annotation of a subset of outputs (a total of 3,016 sentences) belonging to the topic of Politics generated by these four models.Our error annotation focused on eight categories of errors.The error analysis shows that more than 46% of sentences from each of the four models have been errorfree.It uncovered some of the specific classes of errors; for example, WORD errors (mostly verbs and prepositions) are the dominant errors in all four models and are the most complex ones among other errors.NAME (mostly nouns) and NUMBER errors are slightly higher in two of the GeM benchmark models, whereas DATE_DIMENSION and OTHER categories of errors are more common in our Table -to-Text model.This in-depth error analysis is currently guiding us in improving our Table -to-Text model.Input Table data from ToTTo (includes metadata such as Title, Highlighted cells in yellow and their headers): Page Title: List of ambassadors of the United States to Germany