Evaluating Embeddings using Syntax-based Classification Tasks as a Proxy for Parser Performance
Arne Köhn · 2016
Most evaluations for vector space models are semantically motivated, e.g. by measuring how well they capture word similarity.If one is interested in syntax-related downstream applications such as dependency parsing, a syntactically motivated evaluation seems preferable.As we show, the choice of embeddings has a noticeable impact on parser performance.Since evaluating embeddings directly in a parser is costly, we analyze the correlation between the full parsing task and a simple linear classification task as a potential proxy.