$SETL_{onDEMAND}$: Towards an on Demand ETL Approach for Semantic Data Warehouses
Amrit Bhattacharjee, Rudra Pratap Deb Nath · 2024
As the application of Semantic Web technology continues to grow, conducting Online Analytical Processing (OLAP) over Semantic Data Warehouses (SDW) has become an essential task. The Extract-Transform-Load (ETL) process loads data from external sources into an SDW. Generally, OLAP operations function efficiently on static data with minimal changes over time. However, challenges arise when dealing with frequent data changes, such as prices, populations, or social media data, requiring constant updates to the SDW for query processing. Additionally, not all dimensions of the cube may be crucial for answering a query. In this paper, we propose an approach named$SETL_{onDEM \ AND}$, which extracts an OLAP query to determine the required data, fetches the necessary data from respective sources using the ETL pipeline, and then executes the query. This approach additionally enables data retrieval from external data endpoints and seamlessly integrates the outcomes from these external endpoints with local data sources, particularly in cases where the query is federated. We assess the performance, productivity, and quality of$SETL_{onDEM \ AND}$compared to a traditional ETLQ approach.