GitHub's big data adaptor: an eclipse plugin
Ali Sajedi Badashian, Vraj Shah, Eleni Stroulia · Computer Science and Software Engineering · 2015
The of GitHub, the most popular code-sharing platform, fits the characteristics of big data (Volume, Variety and Velocity). To facilitate studies on this huge GitHub volume, the GHTorrent web-site publishes a MYSQL dump of (some) GitHub quarterly. Unfortunately, developers using these published dumps face challenges with respect to the time required to parse and ingest the data, the space required to store it, and the latency of their queries. To help address these challenges, we developed a adaptor as an Eclipse plugin, which efficiently handles this dump. The plugin offers an interactive interface through which users can explore and select any field in any table. After extracting the selected by the user, the parser exports it in easy-to-use spreadsheets. We hope that using this plugin will facilitate further studies on the GitHub as a whole.