Scalable Spatial Analytics and In Situ Query Processing in DaskDB
Suvam Kumar Das, Ronnit Peter, Suprio Ray · 2023
Vast amounts of data are stored in raw data files. Data scientists and practitioners typically use data science frameworks for data analysis on raw data. Among them, Python Pandas library is one of the most popular language-based frameworks. On the other hand, relational databases (RDBMSs) are still widely used for SQL query execution. Before querying, raw data must be loaded into RDBMSs through an ETL process. Conversely, data stored in RDBMSs may need to be exported out or moved into a suitable format to perform complex data analysis. This movement of data adversely affects the time-to-insight.