Secure Image Inference Using Pairwise Activation Functions
Jonas T. Agyepong, Mostafa I. Soliman, Yasutaka Wada, Keiji Richard Kimura, Ahmed El-Mahdy · IEEE Access · 2021
Amongst most artificial neural networks, there is considerable reliance on (raw) data in their use, thus raising data privacy and security issues. A homomorphic encryption scheme affords the performing of computations on encrypted data though it has some challenges, including the lack of support for performing computations using transcendental functions such as logarithm and exponentiation, which exists in some of the typical activation functions in neural networks. Polynomial approximation, though can be used to derive polynomials as an approximation to these activation functions as homomorphic encryption scheme favours mostly addition and multiplication operations. While the literature has considered forming new activation functions as pairwise multiplication of well-known activation functions, the design space is mostly unexplored. Therefore this paper explores the design space of such pairwise, multiplied activation functions and their application in homomorphic image inferencing or prediction using the widely popular MNIST and CIFAR-10 benchmark datasets. Moreover, it analyses corresponding curve fitting parameters (range and degree), homomorphic-friendly pooling methods, and optimization methods in the ciphertext domain to avoid incurring huge computation costs via bootstrapping but not compromising accuracy. Results show new activation function combinations yielding similar or better results in ciphertext as compared to the ones in plaintext.