Branchy-TEE: Deep Learning Security Inference Acceleration Using Trusted Execution Environment
Yulong Wang, Kai Ying Deng, Fanzhi Meng, Zhi Chen, Mingyong Yin, Run Yang · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · 2023
Deep Learning as a Service (DLaaS) has become a remarkable trend in modern data-driven online services.Both data holders and service providers need to build on trust in thirdparty cloud infrastructure platforms.However, once the trust is broken, data holders' sensitive data and service providers' intellectual property rights will face significant security and privacy risks.In this paper, we propose a secure and efficient inference framework for deep learning in untrustworthy cloud platforms, termed Branchy-TEE, which aims to protect the confidentiality and integrity of data and models of multiple participating actors throughout the inference process using the Trusted Execution Environment (TEE).Branchy-TEE dynamically loads the inference network into the TEE on-demand based on early-exit mechanism, expecting to break the hardware performance bottleneck of the TEE.Moreover, a joint training method based on knowledge distillation for multi-exit networks is proposed, by flowing "knowledge" from the final exit with high accuracy to the early branch exit with lower accuracy.Finally, the effectiveness and efficiency of Branchy-TEE are verified through extensive experiments in real environments, while achieving an optimal balance between performance and hardware resources.