Machine learning algorithms based on deep Neural Networks (NN) have achieved remarkable results and are being extensively used in different domains. On the other hand, with increasing growth of cloud services, several Machine Learning as a Service (MLaaS) are offered where training and deploying machine learning models are performed on cloud providers’ infrastructure. However, machine learning algorithms require access to the raw data which is often privacy sensitive and can create potential security and privacy risks. To address this issue, we present CryptoDL, a framework that develops new techniques to provide solutions for applying deep neural network algorithms to encrypted data. In this paper, we provide the theoretical foundation for implementing deep neural network algorithms in encrypted domain and develop techniques to adopt neural networks within practical limitations of current homomorphic encryption schemes. We show that it is feasible and practical to train neural networks using encrypted data and to make encrypted predictions, and also return the predictions in an encrypted form. We demonstrate applicability of the proposed CryptoDL using a large number of datasets and evaluate its performance. The empirical results show that it provides accurate privacy-preserving training and classification.
 M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov. Deep Learning with Differential Privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16), Vienna, Austria. pp 308-318.
 L. J. M. Aslett, P. M. Esperança, C. C. Holmes. Encrypted statistical machine learning: new privacy preserving methods. CoRR abs/1508.06845 (2015).
 L. J. M. Aslett, P. M. Esperanca, C. Holmes. A review of homomorphic encryption and software tools for encrypted statistical machine learning. Tech. rep., University of Oxford, 2015.
 K. Atkinson, W. Han. Theoretical Numerical Analysis: A Functional Analysis Framework. Texts in Applied Mathematics. Springer New York, 2009.
 R. Bost, R. A. Popa, S. TU, S. GOLDWASSER. Machine learning classification over encrypted data. In 22nd Annual Network and Distributed System Security Symposium, NDSS, San Diego, California, USA (2015).
 D. Boneh, Wu. David, H Jacob. Using homomorphic encryption for large scale statistical analysis. 2012.
 Y. Chen, G. Gong. Integer arithmetic over ciphertext and homomorphic data aggregation. In Communications and Network Security (CNS), IEEE Conference on (Sept 2015), pp. 628-632.
 T. S. Developers. SageMath, the Sage Mathematics Software System (Version 7.1), 2016. http://www.sagemath.org.
 N. Dowlin, R. Gilad-Bachrach, K. Laine, K. Lauther, M. Naehrig, J. Wernsing. Manual for using homomorphic encryption for bioinformatics. Tech. Rep. MSR-TR-2015-87, November 2015.
 N. Dowlin, R. Gilad-Bachrach, K. Laine, K. L. M. Naehrig, J. Wersning. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. Proceedings of The 33rd International Conference on Machine Learning, in PMLR 48:201-210.
 T. Elgamal. A public key cryptosystem and a signature scheme based on discrete logarithms. In Proceedings of CRYPTO 84 on Advances in Cryptology (New York, NY, USA, 1985), Springer-Verlag New York, Inc., pp. 10-18.
 M. Fredrikson, S. Jha, T. Ristenpart. Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures. ACM Conference on Computer and Communications Security 2015, Denver, CO, USA, October 12-6, 2015, 1322-1333.
 C. Gentry. A Fully Homomorphic Encryption Scheme. PhD thesis, Stanford, CA, USA, 2009. AAI3382729.
 M. Ghasemi. https://github.com/mghasemi/pyProximation.
 Google Prediction API, https://cloud.google.com/prediction/, Accessed: 2017-02-20.
 S. Goldwasser, S. Micali. Probabilistic encryption & how to play mental poker keeping secret all partial information. In Proceedings of the Fourteenth Annual ACM Symposium on Theory of Computing (New York, NY, USA, 1982), STOC ’82, ACM, pp. 365-377.
 T. Geaepel, K. Lauter, M. Naehrig. ML confidential: Machine learning on encrypted data. In Proceedings of the 15th International Conference on Information Security and Cryptology (Berlin, Heidelberg, 2013), ICISC’12, Springer-Verlag, pp. 1-21.
 S. Halvei, V. Shoup. Algorithms in HElib. In Advances in Cryptology - CRYPTO - 34th Annual Cryptology Conference, Santa Barbara, CA, USA, Proceedings, Part I (2014), pp. 554- 571.
 E. Hesamifard, H. Takabi, M. Ghasemi. CryptoDL: Towards Deep Learning over Encrypted Data. Annual Computer Security Applications Conference (ACSAC 2016), Los Angeles, California, USA.
 E. Hesamifard, H. Takabi, M. Ghasemi, C. Jones. Privacypreserving Machine Learning in Cloud. In Proceedings of the 2017 on Cloud Computing Security Workshop (CCSW ’17), ACM, New York, NY, USA, pp. 39-43.
 E. Hesamifard, H. Takabi, M. Ghasemi. CryptoDL: Deep Neural Networks over Encrypted Data. arXiv preprint arXiv:1711.05189, 2017.
 N. Islam, W. Puech, K. HAYAT, R. BROUZET. Application of Homomorphism to Secure Image Sharing. Optics Communications 284, 19 (Sept. 2011), 4412-4429.
 L. Jian, J. Mika, L. Yao, A. N. Oblivious Neural Network Predictions via MiniONN Transformations. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS ’17, 2017. Dallas, Texas, USA.
 B. Karlik, A. V. Olgac. Performance analysis of various activation functions in generalized mlp architectures of neural networks. International Journal of Artificial Intelligence and Expert Systems 1, 4 (2011), 111-122.
 M. Kim, K. E. Lauther. Private genome analysis through homomorphic encryption. IACR Cryptology ePrint Archive 2015 (2015), 965.
 A. Krizhevsky, V. Nair, G. Hinton. CIFAR-10 (Canadian Institute for Advanced Research), http://www.cs.toronto.edu/kriz/cifar.html
 M. LICHMAN, UCI machine learning repository, 2013.
 F. Liu, W. K. Ng, W. Zhang. Secure scalar product for bigdata in MapReduce. In Big Data Computing Service and Applications (BigDataService), IEEE First International Conference on (March 2015), pp. 120-129.
 R. Livni, S. Shalev-Shwartz, O. Shamir. On the computational efficiency of training neural networks. CoRR abs/1410.1141 (2014).
 W. J. LU, Y. YAMADA, J. SAKUMA. Privacy-preserving genome-wide association studies on cloud environment using fully homomorphic encryption. BMC Medical Informatics and Decision Making 15, 5 (2015), 1-8.
 Microsoft Azure Machine Learning, https://azure.microsoft.com/en-us/services/machinelearning/, Accessed: 2017-02-20.
 P. Mohassel, Y, Zhang. SecureML: A System for Scalable Privacy-Preserving Machine Learning. IACR Cryptology ePrint Archive, 2017.
 P. PAILLIER. Public-key cryptosystems based on composite degree residuosity classes. In Proceedings of the 17th International Conference on Theory and Application of Cryptographic Techniques (Berlin, Heidelberg, 1999), EUROCRYPT’ 99, Springer-Verlag, pp. 223-238.
 P. PILOTTE. Neural Network Toolbox, 2016.
 S. RANE, W. SUN, A. VETRO. Secure distortion computation among untrusting parties using homomorphic encryption. In ICIP (2009), IEEE, pp. 1485-1488.
 M. S. Riazi, C. Weinert, O. Tkachenko, E. M. Songhori, T. Schneider, F. Koushanfar. Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications. Cryptology ePrint Archive, Report 2017/1164, 2017.
 R. Rivest, L. Adleman, M. Dertouzos. On data banks and privacy homomorphisms. In Foundations on Secure Computation, Academia Press (1978), pp. 169-179.
 Bita. D. Rouhani, M. Sadegh Riazi, F. Koushanfar. DeepSecure: Scalable Provably-Secure Deep Learning. CoRR abs/1705.08963 (2017). http://arxiv.org/abs/1705.08963
 R. Shokri, V. Shmatikov. Privacy-Preserving Deep Learning. Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security (CCS ’15). Denver, Colorado, USA. pp 1310-1321.
 R. Shokri, M. Stronati, V. Shmatikov. Membership Inference Attacks against Machine Learning Models. arXiv preprint. arXiv: abs/1610.05820 (2016).
 T. Shortell, A. Shokoufandeh.Secure signal processing using fully homomorphic encryption. In Advanced Concepts for Intelligent Vision Systems - 16th International Conference, ACIVS, Catania, Italy, October 26-29, Proceedings (2015), pp. 93-104.
 H. Takabi, E. Hesamifard, M. Ghasemi. Privacy Preserving Multi-party Machine Learning with Homomorphic Encryption. Private Multi-Party Machine Learning, NIPS 2016 Workshop, Barcelona, Spain.
 M. Togan, C. Plesca. Comparison-based computations over fully homomorphic encrypted data. In Communications (COMM), 10th International Conference on (May 2014), pp. 1-6.
 F. Tramèr, F. Zhang, A. Juels, M.K. Reiter, T. Ristenpart. Stealing Machine Learning Models via Prediction APIs. 25th USENIX Security Symposium, USENIX Security 16, Austin, TX, USA, August 10-12, 2016. 601-618.
 T. Veugen. Encrypted integer division and secure comparison. Int. J. Appl. Cryptol. 3, 2 (June 2014), 166-180.
 P. Xie, M. Bilenko, T. Finley, R. Gilda-Bachrach, K. E. Lauter, M. Naehrig. Crypto-nets: Neural networks over encrypted data. CoRR abs/1412.6181 (2014).
 Y. Xu, Orthogonal polynomials of several variables. Encyclopedia of Mathematics and its Applications 81 (2001).
 L. Yann, C. Corinna, C. J. C. Burges. MNIST handwritten digit database. 2010.
 J. Yuan, S. Yu. Privacy Preserving Back-Propagation Learning Made Practical with Cloud Computing. Security and Privacy in Communication Networks: 8th International ICST Conference, SecureComm 2012, Padua, Italy, September 3-5, 2012. Revised Selected Papers. 2013, Springer Berlin Heidelberg, Berlin, Heidelberg, 292-309.
 J. Zhan. Using Homomorphic Encryption For Privacy- Preserving Collaborative Decision Tree Classification. 2007 IEEE Symposium on Computational Intelligence and Data Mining, 637-645.
 Y. Zhang, W. Dai, X. Jiang, H. Xiong, S. Wang. Foresee: Fully outsourced secure genome study based on homomorphic encryption. BMC Medical Informatics and Decision Making 15, 5 (2015), 1-11. Yuchen Zhang, Wenrui Dai contributed equally to this work.