ropensci/arkdb: arkdb: Archive and Unarchive Databases Using Flat Files
Carl Boettiger · Zenodo (CERN European Organization for Nuclear Research) · 2018
The goal of arkdb is to provide a convienent way to move data from large compressed text files (tsv, csv, etc) into any DBI-compliant database connection (e.g. MYSQL, Postgres, SQLite; see DBI), and move tables out of such databases into text files. The key feature of arkdb is that files are moved between databases and text files in chunks of a fixed size, allowing the package functions to work with tables that would be much to large to read into memory all at once. v0.0.5 Changes ark()'s default keep-open method would cut off header names for Postgres connections (due to variation in the behavior of SQL queries with LIMIT 0.) The issue is now resolved by accessing the header in a more robust, general way.