PROCESSING AND STORAGE OF DIFFERENT DATA WITH THE HELP OF BI-TECHNOLOGIES
Nataliya I. Boyko · 2022
Conditions for the development of modern information space indicate the need to process large amounts of structured, poorly structured and unstructured data, which are in heterogeneous sources. The study describes the approaches, models, methods and tools for building data warehouses, formulates the purpose and objectives of the study, and the concept of building corporate repositories. This section discusses the input and output data features that the system will process. The information model of OLTP systems and data warehouses is also designed; logical essences and their business needs are described. The input data will be the result of the OLTP system, which simulates the operation of the online store. Then, by various means, different amounts of data will be transferred with specific processing to the data warehouse, from the structure of which it will be possible to obtain the original data for further analysis. This section explores the various tools for implementing the system describes their advantages and disadvantages. SQLServer was chosen as the data processing mechanism and DatabaseEngine / SSIS packages as the tools for forming the integration layer and ETL processes. The necessary software and hardware have been included. The object of the study is the process of migrating data from a database that works directly with a business program (OLTP system) to a data warehouse (OLAP system) for further archival storage and analysis. The work aims to create a relational database that will simulate the work of the online store, create an appropriate data warehouse and study the speed of the ETL process of data transfer to the data warehouse in different ways. The main tasks for the realization of the research goal should include construction of a relational database and its filling with suboptimal data; construction of a data warehouse; creation of ETL process with the help of SSIS packages; design of ETL process using T-SQL; comparison of the obtained results and conclusions. We also considered the software implementation, which conducted a study of the speed of data transfer using two selected tools and felt some functionality of the system so that the user interacted correctly with it. To achieve this goal, developing a data warehouse on a hybrid approach is necessary and configuring the ETL process between the relational database and the data warehouse using SSIS packages. To perform experiments, you need to compare data transfer efficiency on tables of different dimensions. As a result of the study, a three-tier system was created, consisting of an OLTP system as a transactional layer, ETL processes using DatabaseEngine and SSIS packages, as an integration layer and a data warehouse built on a hybrid principle as an analytical layer. The implemented system is analogous to the latest self-service system, as it can provide its business needs without third-party funds.