Surprise Languages: Rapid-Response Cross-Language IR
Douglas W. Oard, Marine Jacinthe Carpuat, Petra Galuščáková, Joseph Barrow, Suraj Nair, Xing Hua Niu, Han-Chin Shing, Weijia Xu, Elena Zotkina, Kathleen R. McKeown, Smaranda Muresan, Efsun Sarioglu Kayi, Ramy Eskander, Chris Kedzie, Yan Virin, Dragomir Radev, Rui Zhang, Mark Gales, Anton Ragni, Kenneth Heafield · Edinburgh Research Explorer (University of Edinburgh) · 2019
Sixteen years ago, the first "surprise language exercise" was conducted, in Cebuano. The evaluation goal of a surprise language exercise is to learn how well systems for a new language can be quickly built. This paper briefly reviews the history of surprise language exercises. Some details from the most recent surprise language exercise, in Lithuanian, are included to help to illustrate how the state of the art has advanced over this period.