A Data Analysis Privacy Regulation Compliance Scheme for Lakehouse
Chiyuan Ma, Xiangcheng Hu · 2023
To meet the diverse data storage and analysis needs in the Internet of Things era, businesses embrace the lakehouse approach, a hybrid deployment of data lakes and data warehouses on a single platform. Data consumers leverage data mining techniques through open APIs to explore data’s untapped potential. However, concerns arise regarding compliant data access and utilization. While privacy regulations like the General Data Protection Regulation (GDPR) offer conceptual guidance, their technical implementations remain vague. This paper proposes a privacy regulation compliance framework specific to lakehouse data analysis. By introducing a compliance verification layer between the analysis and processing layers, the scheme enables regulatory adherence. The utilization of Trusted Execution Environments (TEEs) guarantees verification of analysis requests, with blockchain serving as a storage medium for results. To mitigate unauthorized data analysis, we introduce a reputation-based punishment mechanism. Experimental results demonstrate the scheme’s feasibility.