Which argument is more convincing? Analyzing and predicting convincingness of Web arguments using bidirectional LSTM
Ivan Habernal, Iryna Gurevych · 2016
We propose a new task in the field of computational argumentation in which we investigate qualitative properties of Web arguments, namely their convincingness.We cast the problem as relation classification, where a pair of arguments having the same stance to the same prompt is judged.We annotate a large datasets of 16k pairs of arguments over 32 topics and investigate whether the relation "A is more convincing than B" exhibits properties of total ordering; these findings are used as global constraints for cleaning the crowdsourced data.We propose two tasks: (1) predicting which argument from an argument pair is more convincing and (2) ranking all arguments to the topic based on their convincingness.We experiment with feature-rich SVM and bidirectional LSTM and obtain 0.76-0.78accuracy and 0.35-0.40Spearman's correlation in a cross-topic evaluation.We release the newly created corpus UKPConvArg1 and the experimental software under open licenses.