Illuminating the Dark or how to recover what should not be seen in FE-based classifiers

Sergiu Carpov 1 , Caroline Fontaine 2 , Damien Ligier 3  and Renaud Sirdey 4
  • 1 , France
  • 2 Université Paris-Saclay, , France
  • 3 , Paris, France
  • 4 , Gifsur-Yvette, France


Classification algorithms/tools become more and more powerful and pervasive. Yet, for some use cases, it is necessary to be able to protect data privacy while benefiting from the functionalities they provide. Among the tools that may be used to ensure such privacy, we are focusing in this paper on functional encryption. These relatively new cryptographic primitives enable the evaluation of functions over encrypted inputs, outputting cleartext results. Theoretically, this property makes them well-suited to process classification over encrypted data in a privacy by design’ rationale, enabling to perform the classification algorithm over encrypted inputs (i.e. without knowing the inputs) while only getting the input classes as a result in the clear.

In this paper, we study the security and privacy issues of classifiers using today practical functional encryption schemes. We provide an analysis of the information leakage about the input data that are processed in the encrypted domain with state-of-the-art functional encryption schemes. This study, based on experiments ran on MNIST and Census Income datasets, shows that neural networks are able to partially recover information that should have been kept secret. Hence, great care should be taken when using the currently available functional encryption schemes to build privacy-preserving classification services. It should be emphasized that this work does not attack the cryptographic security of functional encryption schemes, it rather warns the community against the fact that they should be used with caution for some use cases and that the current state-ofthe-art may lead to some operational weaknesses that could be mitigated in the future once more powerful functional encryption schemes are available.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] Michel Abdalla, Florian Bourse, Angelo De Caro, and David Pointcheval. Simple functional encryption schemes for inner products. In IACR International Workshop on Public Key Cryptography, pages 733–751. Springer, 2015.

  • [2] Rakesh Agrawal and Ramakrishnan Srikant. Privacy-preserving data mining, volume 29. ACM, 2000.

  • [3] Shweta Agrawal, Benoît Libert, and Damien Stehlé. Fully secure functional encryption for inner products, from standard assumptions. In Annual Cryptology Conference, pages 333–362. Springer, 2016.

  • [4] Giuseppe Ateniese, Giovanni Felici, Luigi V Mancini, Angelo Spognardi, Antonio Villani, and Domenico Vitali. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. arXiv preprint arXiv:1306.4447, 2013.

  • [5] Michael Backes, Pascal Berrang, Mathias Humbert, and Praveen Manoharan. Membership privacy in microrna-based studies. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 319–330. ACM, 2016.

  • [6] Carmen Elisabetta Zaira Baltico, Dario Catalano, Dario Fiore, and Romain Gay. Practical functional encryption for quadratic functions with applications to predicate encryption. In Annual International Cryptology Conference, pages 67–98. Springer, 2017.

  • [7] Dan Boneh and Matt Franklin. Identity-based encryption from the weil pairing. In Advances in Cryptology – CRYPTO 2001, pages 213–229. Springer, 2001.

  • [8] Dan Boneh, Amit Sahai, and Brent Waters. Functional encryption: Definitions and challenges. In Theory of Cryptography Conference, pages 253–273. Springer, 2011.

  • [9] Léon Bottou, Corinna Cortes, John S Denker, Harris Drucker, Isabelle Guyon, Lawrence D Jackel, Yann LeCun, Urs A Muller, Edward Sackinger, Patrice Simard, et al. Comparison of classifier methods: a case study in handwritten digit recognition. In International conference on pattern recognition, pages 77–77. IEEE Computer Society Press, 1994.

  • [10] David Cash, Paul Grubbs, Jason Perry, and Thomas Risten-part. Leakage-abuse attacks against searchable encryption. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 668–679. ACM, 2015.

  • [11] François Chollet et al. Keras., 2015.

  • [12] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.

  • [13] Edouard Dufour Sans, Romain Gay, and David Pointcheval. Reading in the dark: Classifying encrypted digits with functional encryption. Cryptology ePrint Archive, Report 2018/206, 2018.

  • [14] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322–1333. ACM, 2015.

  • [15] Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In USENIX Security Symposium, pages 17–32, 2014.

  • [16] Sanjam Garg, Craig Gentry, Shai Halevi, Mariana Raykova, Amit Sahai, and Brent Waters. Candidate indistinguishability obfuscation and functional encryption for all circuits. In Foundations of Computer Science (FOCS), 2013 IEEE 54th Annual Symposium on, pages 40–49. IEEE, 2013.

  • [17] Shafi Goldwasser, Yael Kalai, Raluca Ada Popa, Vinod Vaikuntanathan, and Nickolai Zeldovich. Reusable garbled circuits and succinct functional encryption. In Proceedings of the forty-fifth annual ACM symposium on Theory of computing, pages 555–564. ACM, 2013.

  • [18] Paul Grubbs, Kevin Sekniqi, Vincent Bindschaedler, Muhammad Naveed, and Thomas Ristenpart. Leakage-abuse attacks against order-revealing encryption. In 2017 IEEE Symposium on Security and Privacy (SP), pages 655–672. IEEE, 2017.

  • [19] Nils Homer, Szabolcs Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling, John V Pearson, Dietrich A Stephan, Stanley F Nelson, and David W Craig. Resolving individuals contributing trace amounts of dna to highly complex mixtures using high-density snp genotyping microarrays. PLoS genetics, 4(8):e1000167, 2008.

  • [20] M. Hutter. Universal Artificial Intelligence|Sequential Decisions Based on Algorithmic Probability. Springer, 2005.

  • [21] Marcus Hutter. Universal artificial intelligence: Sequential decisions based on algorithmic probability. Springer Science & Business Media, 2004.

  • [22] Marcus Hutter. On the foundations of universal sequence prediction. In International Conference on Theory and Applications of Models of Computation, pages 408–420. Springer, 2006.

  • [23] Jonathan Katz, Amit Sahai, and Brent Waters. Predicate encryption supporting disjunctions, polynomial equations, and inner products. In Advances in Cryptology – EURO-CRYPT 2008, pages 146–162. Springer, 2008.

  • [24] Ron Kohavi. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pages 202–207. AAAI Press, 1996.

  • [25] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, volume 86, pages 2278–2324. IEEE, 1998.

  • [26] Yann LeCun, Corinna Cortes, and Christopher J.C. Burges. The MNIST Database.

  • [27] Shane Legg. Is there an elegant universal theory of prediction? In International Conference on Algorithmic Learning Theory, pages 274–287. Springer, 2006.

  • [28] Ming Li, Shucheng Yu, Yao Zheng, Kui Ren, and Wenjing Lou. Scalable and secure sharing of personal health records in cloud computing using attribute-based encryption. IEEE transactions on parallel and distributed systems, 24(1):131–143, 2013.

  • [29] Damien Ligier, Sergiu Carpov, Caroline Fontaine, and Renaud Sirdey. Information leakage analysis of inner-product functional encryption based data classification. In PST’17: 15th International Conference on Privacy, Security and Trust. IEEE, 2017.

  • [30] Damien Ligier, Sergiu Carpov, Caroline Fontaine, and Renaud Sirdey. Privacy preserving data classification using inner-product functional encryption. In ICISSP, pages 423–430, 2017.

  • [31] Yehuda Lindell and Benny Pinkas. Privacy preserving data mining. In Annual International Cryptology Conference, pages 36–54. Springer, 2000.

  • [32] Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Stand-alone and federated learning under passive and active white-box inference attacks. arXiv preprint arXiv:1812.00910, 2018.

  • [33] Tatsuaki Okamoto and Katsuyuki Takashima. Hierarchical predicate encryption for inner-products. In International Conference on the Theory and Application of Cryptology and Information Security, pages 214–231. Springer, 2009.

  • [34] Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. Towards the science of security and privacy in machine learning. arXiv preprint arXiv:1611.03814, 2016.

  • [35] Bryan Parno, Mariana Raykova, and Vinod Vaikuntanathan. How to delegate and verify in public: Verifiable computation from attribute-based encryption. In Theory of Cryptography Conference, pages 422–439. Springer, 2012.

  • [36] Théo Ryffel, Edouard Dufour-Sans, Romain Gay, Francis Bach, and David Pointcheval. Partially Encrypted Machine Learning using Functional Encryption. In NeurIPS 2019 -Thirty-third Conference on Neural Information Processing Systems, Advances in Neural Information Processing Systems, Vancouver, Canada, December 2019.

  • [37] Amit Sahai and Brent Waters. Fuzzy identity-based encryption. In Annual International Conference on the Theory and Applications of Cryptographic Techniques, pages 457–473. Springer, 2005.

  • [38] Sriram Sankararaman, Guillaume Obozinski, Michael I Jordan, and Eran Halperin. Genomic privacy and limits of individual detection in a pool. Nature genetics, 41(9):965, 2009.

  • [39] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In Security and Privacy (SP), 2017 IEEE Symposium on, pages 3–18. IEEE, 2017.

  • [40] Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. In USENIX Security Symposium, pages 601–618, 2016.

  • [41] Vassilios S Verykios, Elisa Bertino, Igor Nai Fovino, Loredana Parasiliti Provenza, Yucel Saygin, and Yannis Theodoridis. State-of-the-art in privacy preserving data mining. ACM Sigmod Record, 33(1):50–57, 2004.

  • [42] Brent Waters. Efficient identity-based encryption without random oracles. In Annual International Conference on the Theory and Applications of Cryptographic Techniques, pages 114–127. Springer, 2005.

  • [43] Brent Waters. Ciphertext-policy attribute-based encryption: An expressive, efficient, and provably secure realization. In International Workshop on Public Key Cryptography, pages 53–70. Springer, 2011.


Journal + Issues