Don’t Interrupt Me While I Type: Inferring Text Entered Through Gesture Typing on Android Keyboards

Open access


We present a new side-channel attack against soft keyboards that support gesture typing on Android smartphones. An application without any special permissions can observe the number and timing of the screen hardware interrupts and system-wide software interrupts generated during user input, and analyze this information to make inferences about the text being entered by the user. System-wide information is usually considered less sensitive than app-specific information, but we provide concrete evidence that this may be mistaken. Our attack applies to all Android versions, including Android M where the SELinux policy is tightened.

We present a novel application of a recurrent neural network as our classifier to infer text. We evaluate our attack against the “Google Keyboard” on Nexus 5 phones and use a real-world chat corpus in all our experiments. Our evaluation considers two scenarios. First, we demonstrate that we can correctly detect a set of pre-defined “sentences of interest” (with at least 6 words) with 70% recall and 60% precision. Second, we identify the authors of a set of anonymous messages posted on a messaging board. We find that even if the messages contain the same number of words, we correctly re-identify the author more than 97% of the time for a set of up to 35 sentences.

Our study demonstrates a new way in which system-wide resources can be a threat to user privacy. We investigate the effect of rate limiting as a countermeasure but find that determining a proper rate is error-prone and fails in subtle cases. We conclude that real-time interrupt information should be made inaccessible, perhaps via a tighter SELinux policy in the next Android version.

[1] A. T. Ozcan, C. Gemicioglu, K. Onarlioglu, M. Weissbacher, C. Mulliner, W. Robertson, and E. Kirda, “BabelCrypt: The Universal Encryption Layer for Mobile Messaging Applications,” in Financial Cryptography and Data Security (FC), 01 2015.

[2] K. Zhang and X. Wang, “Peeping tom in the neighborhood: Keystroke eavesdropping on multi-user systems,” in Proceedings of the 18th Conference on USENIX Security Symposium, SSYM’09, (Berkeley, CA, USA), pp. 17-32, USENIX Association, 2009.

[3] Q. A. Chen, Z. Qian, and Z. M. Mao, “Peeking into your app without actually seeing it: UI state inference and novel android attacks,” in Proceedings of the 23rd USENIX Security Symposium, San Diego, CA, USA, August 20-22, 2014., pp. 1037-1052, 2014.

[4] X. Zhou, S. Demetriou, D. He, M. Naveed, X. Pan, X. Wang, C. A. Gunter, and K. Nahrstedt, “Identity, lo cation, disease and more: Inferring your secrets from android public resources,” in Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, CCS ’13, (New York, NY, USA), pp. 1017-1028, ACM, 2013.

[5] A. Savitzky and M. J. E. Golay, “Smoothing and Differentiation of Data by Simplified Least Squares Procedures.,” Anal. Chem., vol. 36, pp. 1627-1639, July 1964.

[6] T. Mikolov, M. Karafiát, L. Burget, J. Cernock`y, and S. Khudanpur, “Recurrent neural network based language model.,” in INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September 26-30, 2010, pp. 1045-1048, 2010.

[7] T. Mikolov, S. Kombrink, L. Burget, J. H. Cernock`y, and S. Khudanpur, “Extensions of recurrent neural network language model,” in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pp. 5528-5531, IEEE, 2011.

[8] C. D. Manning and H. Schütze, Foundations of Statistical Natural Language Processing. Cambridge, MA, USA: MIT Press, 1999.

[9] J. L. Elman, “Finding structure in time,” Cognitive science, vol. 14, no. 2, pp. 179-211, 1990.

[10] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Cognitive modeling, vol. 5, no. 3, p. 1, 1988.

[11] P. J. Werbos, “Backpropagation through time: what it does and how to do it,” Proceedings of the IEEE, vol. 78, no. 10, pp. 1550-1560, 1990.

[12] “Android apps in sheep’s clothing.”

[13] “”

[14] “Activitymanager.”

[15] E. N. Forsythand and C. H. Martell, “Lexical and discourse analysis of online chat dialog,” in Proceedings of the International Conference on Semantic Computing, ICSC ’07, (Washington, DC, USA), pp. 19-26, IEEE Computer Society, 2007.

[16] S. Bird, E. Klein, and E. Loper, Natural Language Processing with Python. O’Reilly Media, Inc., 1st ed., 2009.

[17] J. Munkres, “Algorithms for the assignment and transportation problems,” 1957.

[18] N. Zhang, K. Yuan, M. Naveed, X. Zhou, and X. Wang, “Leave me alone: App-level protection against runtime information gathering on android,” 2015.

[19] Y. Michalevsky, G. Nakibly, A. Schulman, and D. Boneh, “Powerspy: Location tracking using mobile device power analysis,” arXiv preprint arXiv:1502.03182, 2015.

[20] L. Simon and R. Anderson, “Pin skimmer: Inferring pins through the camera and microphone,” in Proceedings of the Third ACM Workshop on Security and Privacy in Smartphones & Mobile Devices, SPSM ’13, (New York, NY, USA), pp. 67-78, ACM, 2013.

[21] P. C. Kocher, J. Jaffe, and B. Jun, “Differential power analysis,” in Proceedings of the 19th Annual International Cryptology Conference on Advances in Cryptology, CRYPTO ’99, (London, UK, UK), pp. 388-397, Springer-Verlag, 1999.

[22] P. C. Kocher, “Timing attacks on implementations of diffiehellman, rsa, dss, and other systems,” in Proceedings of the 16th Annual International Cryptology Conference on Advances in Cryptology, CRYPTO ’96, (London, UK, UK), pp. 104-113, Springer-Verlag, 1996.

[23] D. A. Osvik, A. Shamir, and E. Tromer, “Cache attacks and countermeasures: The case of aes,” in Proceedings of the 2006 The Cryptographers’ Track at the RSA Conference on Topics in Cryptology, CT-RSA’06, (Berlin, Heidelberg), pp. 1-20, Springer-Verlag, 2006.

[24] D. J. Bernstein, “Cache-timing attacks on aes,” tech. rep., 2005.

[25] M. Vuagnoux and S. Pasini, “Compromising electromagnetic emanations of wired and wireless keyboards.,” in USENIX security symposium, pp. 1-16, 2009.

[26] L. Zhuang, F. Zhou, and J. D. Tygar, “Keyboard acoustic emanations revisited,” ACM Transactions on Information and System Security (TISSEC), vol. 13, no. 1, p. 3, 2009.

[27] M. Backes, M. Dürmuth, S. Gerling, M. Pinkal, and C. Sporleder, “Acoustic side-channel attacks on printers.,” in USENIX Security Symposium, pp. 307-322, 2010.

[28] J. Mäntyjärvi, M. Lindholm, E. Vildjiounaite, S. marja Mäkelä, and H. Ailisto, “Identifying users of portable devices from gait pattern with accelerometers,” in in IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005.

[29] Y. Michalevsky, D. Boneh, and G. Nakibly, “Gyrophone: Recognizing speech from gyroscope signals,” in Proceedings of the 23rd USENIX Conference on Security Symposium, SEC’14, (Berkeley, CA, USA), pp. 1053-1067, USENIX Association, 2014.

[30] S. Nawaz and C. Mascolo, “Mining users’ significant driving routes with low-power sensors,” in Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems, SenSys ’14, (New York, NY, USA), pp. 236-250, ACM, 2014.

[31] Z. Xu, K. Bai, and S. Zhu, “Taplogger: Inferring user inputs on smartphone touchscreens using on-board motion sensors,” in Proceedings of the fifth ACM conference on Security and Privacy in Wireless and Mobile Networks, pp. 113-124, ACM, 2012.

[32] L. Cai and H. Chen, “Touchlogger: Inferring keystrokes on touch screen from smartphone motion.,” in HotSec, 2011.

[33] E. Miluzzo, A. Varshavsky, S. Balakrishnan, and R. R. Choudhury, “Tapprints: your finger taps have fingerprints,” in Proceedings of the 10th international conference on Mobile systems, applications, and services, pp. 323-336, ACM, 2012.

[34] A. J. Aviv, B. Sapp, M. Blaze, and J. M. Smith, “Practicality of accelerometer side channels on smartphones,” in Proceedings of the 28th Annual Computer Security Applications Conference, pp. 41-50, ACM, 2012.

[35] S. Dey, N. Roy, W. Xu, R. R. Choudhury, and S. Nelakuditi, “Accelprint: Imperfections of accelerometers make smartphones trackable,” in Proceedings of the Network and Distributed System Security Symposium (NDSS), 2014.

[36] P. Marquardt, A. Verma, H. Carter, and P. Traynor, “(sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers,” in Proceedings of the 18th ACM conference on Computer and communications security, pp. 551-562, ACM, 2011.

[37] J. Cache, “Fingerprinting 802.11 Implementations via Statistical Analysis of the Duration Field,” tech. rep., 2006.

[38] “Nmap security scanner.” Accessed: 2015-07-31.

[39] V. C. Perta, M. V. Barbera, and A. Mei, “Exploiting delay patterns for user ips identification in cellular networks,” in Privacy Enhancing Technologies, pp. 224-243, Springer, 2014.

[40] V. Brik, S. Banerjee, M. Gruteser, and S. Oh, “Wireless device identification with radiometric signatures,” in Proceedings of the 14th ACM international conference on Mobile computing and networking, pp. 116-127, ACM, 2008.

[41] T. Stöber, M. Frank, J. Schmitt, and I. Martinovic, “Who do you sync you are?: smartphone fingerprinting via application behaviour,” in Proceedings of the sixth ACM conference on Security and privacy in wireless and mobile networks, pp. 7-12, ACM, 2013.

[42] M. Conti, L. V. Mancini, R. Spolaor, and N. V. Verde, “Can’t you hear me knocking: Identification of user actions on android apps via traffic analysis,” in Proceedings of the 5th ACM Conference on Data and Application Security and Privacy, pp. 297-304, ACM, 2015.

[43] S. Chen, R. Wang, X. Wang, and K. Zhang, “Side-channel leaks in web applications: A reality today, a challenge tomorrow,” in Security and Privacy (SP), 2010 IEEE Symposium on, pp. 191-206, IEEE, 2010.

[44] S. Khattak, L. Simon, and S. J. Murdoch, “Systemization of pluggable transports for censorship resistance,” arXiv preprint arXiv:1412.7448, 2014.

[45] C. V. Wright, L. Ballard, S. E. Coull, F. Monrose, and G. M. Masson, “Spot me if you can: Uncovering spoken phrases in encrypted voip conversations,” in Security and Privacy, 2008. SP 2008. IEEE Symposium on, pp. 35-49, IEEE, 2008.

[46] A. M. White, A. R. Matthews, K. Z. Snow, and F. Monrose, “Phonotactic reconstruction of encrypted voip conversations: Hookt on fon-iks,” in Security and Privacy (SP), 2011 IEEE Symposium on, pp. 3-18, IEEE, 2011.

[47] S. Jana and V. Shmatikov, “Memento: Learning secrets from process footprints,” in Security and Privacy (SP), 2012 IEEE Symposium on, pp. 143-157, IEEE, 2012.

Journal Information

Cited By


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 284 284 43
PDF Downloads 120 120 15