Data Analysis in the Twenty-First Century
A. Goodman, Chandrika Kamath, Kumar · University of North Texas Digital Library (University of North Texas) · 2007
The 21st Century is characterized by complex multidisciplinary problems accompanied by massive datasets. 'We are drowning in data, but starving for knowledge', as the volumes of many commercial, industrial and scientific datasets have exceeded the terabyte range and are approaching petabytes and beyond. Statistical methodology has long been employed to find useful and usable information in data. More recently, data mining has harnessed the power of computer technology to find useful and usable patterns in such massive datasets. Although several data mining journals have joined the established statistical journals, no single journal provides an integrated treatment of statistical analysis methodology and data mining technology, particularly when applied to the solution of practical problems. This absence and the needs expressed above motivated the inauguration of John Wiley's new Journal on Statistical Analysis and Data Mining. The goals of this interdisciplinary journal are to encourage collaborations across disciplines, communication of data mining and statistical techniques to both novices and experts involved in the analysis of data from practical problems, and a principled and productive evaluation of analyses and solutions. The journal specifically encourages submission of works that have statistical rigor in the analysis of data, incorporate the most appropriate algorithms from data mining, and address the needs of applications. Applying data mining algorithms to practical problems is not sufficient, because we need to ensure that the results have a sound statistical basis, lest any decision based on these results lead to a catastrophe. Even data mining algorithms founded on sound statistical analysis are not sufficient, if they cannot solve a practical problem. Finally, employing a statistical analysis on a practical problem is not sufficient, unless it scales up to massive datasets. Statistical analysis and data mining are actually two sides of the sword that is sorely needed to conquer data overload in practical problems.