The GOLM-database standard- a framework for time series data management based on free software
Eichler, Mareike, Till Francke, David Kneis, Dominik E. Reusser · GFZ Data Services · 2010
Monitoring and modeling projects usually involves time series data originating from different sources. Often, file formats, temporal resolution, and metadata documentation rarely adhere to a common standard. As a result, much effort is spent on converting, harmonizing, merging, checking, resampling, and reformatting these data. Moreover, in work groups or during the course of time, these tasks tend to be carried out redundantly and repeatedly, especially when new data becomes available. The resulting duplication of data in various formats strains additional resources. We propose a database structure and complementary scripts for facilitating these tasks. The GOLM- (General Observation and Location Management) framework allows import and storage of time series data of different types while assisting metadata documentation, plausibility checking and harmonization. The imported data can be visually inspected and its coverage among locations and variables may be visualized. Supplementing scripts provides options for data export of selected stations and variables and resampling of the data to the desired temporal resolution. These tools can, for example, be used for generating model input files or reports. Since GOLM fully supports network access, the system can be used efficiently by distributed working groups accessing the same data over the internet. GOLMs database structure and the complementary scripts can easily be customized to specific needs. Any involved software such as MySQL, R, PHP, OpenOffice as well as the scripts for building and using the data base, including documentation, are free for download. GOLM was developed out of the practical requirements of the OPAQUE-project. It has been tested and further refined in the ERANET-CRUE and SESAM projects, which all used GOLM to manage meteorological, hydrological and/or water quality data.