Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC.
Jonathan Huang, Chris Piech, Andy Nguyễn, Leonidas Guibas · 2013
Abstract. In the first offering of Stanford’s Machine Learning Massive Open-Access Online Course (MOOC) there were over a million programming submissions to 42 assignments — a dense sampling of the range of possible solutions. In this paper we map out the syntax and functional similarity of the submissions in order to explore the variation in solutions. While there was a massive number of submissions, there is a much smaller set of unique approaches. This redundancy in student solutions can be leveraged to “force multiply ” teacher feedback. Fig.1. The landscape of solutions for “gradient descent for linear regression ” representing over 40,000 student code submissions with edges drawn between syntactically similar submissions and colors corresponding to performance on a battery of unit tests (red submissions passed all unit tests). 1