Scalable Deep Document / Sequence Reasoning with Cognitive Toolkit
Sayan Pathak, Pengcheng He, William Darling · 2017
Deep Neural Networks (DNNs) have revolutionized the way that machines understand language and have allowed us to create models that answer textual questions, translate pairs of languages, and intelligently compare document corpora. At the heart of these successes lie core techniques that fall into the area of sequence understanding. While powerful, dealing with variable-size sequences in DNNs requires deep understanding and experience in creating such networks, which can be daunting to many scientists and engineers. This tutorial will focus on introducing core concepts, end-to-end recipes, and key innovations facilitated by the cross-platform fully open-source Cognitive Toolkit (formerly called CNTK) with superior scalability (up to 1000 GPUs) for very large data corpora. Specifically, we will present tutorials on basic sequence understanding, intermediate sequence-to-sequence translation (both with and without attention), and the advanced Reasoning Network (ReasoNet) which has achieved industry-leading results in reading comprehension.