Data Acquisition Techniques
Jason Bell · Machine Learning · 2020
This chapter outlines a few techniques for acquiring data, whether that be via page scraping, using Google Sheets to import table data, or using scripting languages to clean up files. Acquiring data is one part of the equation; migrating and transforming it will also be requested at some point. For some jobs, writing a small program or script to import/export data would be fine, but as the volumes grow and the demands from stakeholders get more complex, people need to start looking at alternative tools. Embulk is an open source bulk loading tool. It provides a number of plugins to read, write, and transform data. For example, if a person wanted to read a directory of CSV files, transform them to JSON, and write them to AWS S3, that can be done with Embulk with a single configuration file. If he/she is using the OpenJDK, then it uses version 8 without any issues.