Discriminating between Similar Languages with Word-level Convolutional Neural Networks
Marcelo Criscuolo, Sandra Maria Aluísio · 2017
Discriminating between Similar Languages (DSL) is a challenging task addressed at the VarDial Workshop series.We report on our participation in the DSL shared task with a two-stage system.In the first stage, character n-grams are used to separate language groups, then specialized classifiers distinguish similar language varieties.We have conducted experiments with three system configurations and submitted one run for each.Our main approach is a word-level convolutional neural network (CNN) that learns task-specific vectors with minimal text preprocessing.We also experiment with multi-layer perceptron (MLP) networks and another hybrid configuration.Our best run achieved an accuracy of 90.76%, ranking 8th among 11 participants and getting very close to the system that ranked first (less than 2 points).Even though the CNN model could not achieve the best results, it still makes a viable approach to discriminating between similar languages.