Decision tree-based feature function design in conditional random field applied to error detection of ocean observation data
Yosuke Kamikawaji, Haruki Matsuyama, Ken–ichi Fukui, Shigeki Hosoda, Satoshi Ono · 2016
Globally-covered ocean monitoring system Argo with more than 3,700 floats has been working, and its accumulated big ocean observation data helps many studies such as investigation into climate change mechanism. Since the observed data sometimes involves errors, human experts must visually confirm and revise quality control (QC) labels. However, such manual QC by human experts cannot be performed in some contries. In addition, it is difficult to regularize the quality of the ocean observation data of all over the world because the manual QC depends on human experts' heuristics. Therefore, this paper proposes a method for error detection in Argo observation data using Conditional Random Field (CRF) to realize an automatic QC with high accuracy equal to human experts. This paper also proposes a feature function design method using decision tree learning. Using decision tree allows coping with various types of observation errors without manual work, whereas previous work had to focus on certain error types due to manual labor for feature function design. Experimental results have shown that the proposed method could detect all types of salinity errors with automatically designed features while maintaining the higher accuracy of QC label assignments than the actually operated system in Argo project and a previous method using CRF with SVM.