Don’t Interrupt Me While I Type: Inferring Text Entered Through Gesture Typing on Android Keyboards

Open access


We present a new side-channel attack against soft keyboards that support gesture typing on Android smartphones. An application without any special permissions can observe the number and timing of the screen hardware interrupts and system-wide software interrupts generated during user input, and analyze this information to make inferences about the text being entered by the user. System-wide information is usually considered less sensitive than app-specific information, but we provide concrete evidence that this may be mistaken. Our attack applies to all Android versions, including Android M where the SELinux policy is tightened.

We present a novel application of a recurrent neural network as our classifier to infer text. We evaluate our attack against the “Google Keyboard” on Nexus 5 phones and use a real-world chat corpus in all our experiments. Our evaluation considers two scenarios. First, we demonstrate that we can correctly detect a set of pre-defined “sentences of interest” (with at least 6 words) with 70% recall and 60% precision. Second, we identify the authors of a set of anonymous messages posted on a messaging board. We find that even if the messages contain the same number of words, we correctly re-identify the author more than 97% of the time for a set of up to 35 sentences.

Our study demonstrates a new way in which system-wide resources can be a threat to user privacy. We investigate the effect of rate limiting as a countermeasure but find that determining a proper rate is error-prone and fails in subtle cases. We conclude that real-time interrupt information should be made inaccessible, perhaps via a tighter SELinux policy in the next Android version.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] A. T. Ozcan C. Gemicioglu K. Onarlioglu M. Weissbacher C. Mulliner W. Robertson and E. Kirda “BabelCrypt: The Universal Encryption Layer for Mobile Messaging Applications” in Financial Cryptography and Data Security (FC) 01 2015.

  • [2] K. Zhang and X. Wang “Peeping tom in the neighborhood: Keystroke eavesdropping on multi-user systems” in Proceedings of the 18th Conference on USENIX Security Symposium SSYM’09 (Berkeley CA USA) pp. 17-32 USENIX Association 2009.

  • [3] Q. A. Chen Z. Qian and Z. M. Mao “Peeking into your app without actually seeing it: UI state inference and novel android attacks” in Proceedings of the 23rd USENIX Security Symposium San Diego CA USA August 20-22 2014. pp. 1037-1052 2014.

  • [4] X. Zhou S. Demetriou D. He M. Naveed X. Pan X. Wang C. A. Gunter and K. Nahrstedt “Identity lo cation disease and more: Inferring your secrets from android public resources” in Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security CCS ’13 (New York NY USA) pp. 1017-1028 ACM 2013.

  • [5] A. Savitzky and M. J. E. Golay “Smoothing and Differentiation of Data by Simplified Least Squares Procedures.” Anal. Chem. vol. 36 pp. 1627-1639 July 1964.

  • [6] T. Mikolov M. Karafiát L. Burget J. Cernock`y and S. Khudanpur “Recurrent neural network based language model.” in INTERSPEECH 2010 11th Annual Conference of the International Speech Communication Association Makuhari Chiba Japan September 26-30 2010 pp. 1045-1048 2010.

  • [7] T. Mikolov S. Kombrink L. Burget J. H. Cernock`y and S. Khudanpur “Extensions of recurrent neural network language model” in Acoustics Speech and Signal Processing (ICASSP) 2011 IEEE International Conference on pp. 5528-5531 IEEE 2011.

  • [8] C. D. Manning and H. Schütze Foundations of Statistical Natural Language Processing. Cambridge MA USA: MIT Press 1999.

  • [9] J. L. Elman “Finding structure in time” Cognitive science vol. 14 no. 2 pp. 179-211 1990.

  • [10] D. E. Rumelhart G. E. Hinton and R. J. Williams “Learning representations by back-propagating errors” Cognitive modeling vol. 5 no. 3 p. 1 1988.

  • [11] P. J. Werbos “Backpropagation through time: what it does and how to do it” Proceedings of the IEEE vol. 78 no. 10 pp. 1550-1560 1990.

  • [12] “Android apps in sheep’s clothing.”

  • [13] “”

  • [14] “Activitymanager.”

  • [15] E. N. Forsythand and C. H. Martell “Lexical and discourse analysis of online chat dialog” in Proceedings of the International Conference on Semantic Computing ICSC ’07 (Washington DC USA) pp. 19-26 IEEE Computer Society 2007.

  • [16] S. Bird E. Klein and E. Loper Natural Language Processing with Python. O’Reilly Media Inc. 1st ed. 2009.

  • [17] J. Munkres “Algorithms for the assignment and transportation problems” 1957.

  • [18] N. Zhang K. Yuan M. Naveed X. Zhou and X. Wang “Leave me alone: App-level protection against runtime information gathering on android” 2015.

  • [19] Y. Michalevsky G. Nakibly A. Schulman and D. Boneh “Powerspy: Location tracking using mobile device power analysis” arXiv preprint arXiv:1502.03182 2015.

  • [20] L. Simon and R. Anderson “Pin skimmer: Inferring pins through the camera and microphone” in Proceedings of the Third ACM Workshop on Security and Privacy in Smartphones & Mobile Devices SPSM ’13 (New York NY USA) pp. 67-78 ACM 2013.

  • [21] P. C. Kocher J. Jaffe and B. Jun “Differential power analysis” in Proceedings of the 19th Annual International Cryptology Conference on Advances in Cryptology CRYPTO ’99 (London UK UK) pp. 388-397 Springer-Verlag 1999.

  • [22] P. C. Kocher “Timing attacks on implementations of diffiehellman rsa dss and other systems” in Proceedings of the 16th Annual International Cryptology Conference on Advances in Cryptology CRYPTO ’96 (London UK UK) pp. 104-113 Springer-Verlag 1996.

  • [23] D. A. Osvik A. Shamir and E. Tromer “Cache attacks and countermeasures: The case of aes” in Proceedings of the 2006 The Cryptographers’ Track at the RSA Conference on Topics in Cryptology CT-RSA’06 (Berlin Heidelberg) pp. 1-20 Springer-Verlag 2006.

  • [24] D. J. Bernstein “Cache-timing attacks on aes” tech. rep. 2005.

  • [25] M. Vuagnoux and S. Pasini “Compromising electromagnetic emanations of wired and wireless keyboards.” in USENIX security symposium pp. 1-16 2009.

  • [26] L. Zhuang F. Zhou and J. D. Tygar “Keyboard acoustic emanations revisited” ACM Transactions on Information and System Security (TISSEC) vol. 13 no. 1 p. 3 2009.

  • [27] M. Backes M. Dürmuth S. Gerling M. Pinkal and C. Sporleder “Acoustic side-channel attacks on printers.” in USENIX Security Symposium pp. 307-322 2010.

  • [28] J. Mäntyjärvi M. Lindholm E. Vildjiounaite S. marja Mäkelä and H. Ailisto “Identifying users of portable devices from gait pattern with accelerometers” in in IEEE International Conference on Acoustics Speech and Signal Processing 2005.

  • [29] Y. Michalevsky D. Boneh and G. Nakibly “Gyrophone: Recognizing speech from gyroscope signals” in Proceedings of the 23rd USENIX Conference on Security Symposium SEC’14 (Berkeley CA USA) pp. 1053-1067 USENIX Association 2014.

  • [30] S. Nawaz and C. Mascolo “Mining users’ significant driving routes with low-power sensors” in Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems SenSys ’14 (New York NY USA) pp. 236-250 ACM 2014.

  • [31] Z. Xu K. Bai and S. Zhu “Taplogger: Inferring user inputs on smartphone touchscreens using on-board motion sensors” in Proceedings of the fifth ACM conference on Security and Privacy in Wireless and Mobile Networks pp. 113-124 ACM 2012.

  • [32] L. Cai and H. Chen “Touchlogger: Inferring keystrokes on touch screen from smartphone motion.” in HotSec 2011.

  • [33] E. Miluzzo A. Varshavsky S. Balakrishnan and R. R. Choudhury “Tapprints: your finger taps have fingerprints” in Proceedings of the 10th international conference on Mobile systems applications and services pp. 323-336 ACM 2012.

  • [34] A. J. Aviv B. Sapp M. Blaze and J. M. Smith “Practicality of accelerometer side channels on smartphones” in Proceedings of the 28th Annual Computer Security Applications Conference pp. 41-50 ACM 2012.

  • [35] S. Dey N. Roy W. Xu R. R. Choudhury and S. Nelakuditi “Accelprint: Imperfections of accelerometers make smartphones trackable” in Proceedings of the Network and Distributed System Security Symposium (NDSS) 2014.

  • [36] P. Marquardt A. Verma H. Carter and P. Traynor “(sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers” in Proceedings of the 18th ACM conference on Computer and communications security pp. 551-562 ACM 2011.

  • [37] J. Cache “Fingerprinting 802.11 Implementations via Statistical Analysis of the Duration Field” tech. rep. 2006.

  • [38] “Nmap security scanner.” Accessed: 2015-07-31.

  • [39] V. C. Perta M. V. Barbera and A. Mei “Exploiting delay patterns for user ips identification in cellular networks” in Privacy Enhancing Technologies pp. 224-243 Springer 2014.

  • [40] V. Brik S. Banerjee M. Gruteser and S. Oh “Wireless device identification with radiometric signatures” in Proceedings of the 14th ACM international conference on Mobile computing and networking pp. 116-127 ACM 2008.

  • [41] T. Stöber M. Frank J. Schmitt and I. Martinovic “Who do you sync you are?: smartphone fingerprinting via application behaviour” in Proceedings of the sixth ACM conference on Security and privacy in wireless and mobile networks pp. 7-12 ACM 2013.

  • [42] M. Conti L. V. Mancini R. Spolaor and N. V. Verde “Can’t you hear me knocking: Identification of user actions on android apps via traffic analysis” in Proceedings of the 5th ACM Conference on Data and Application Security and Privacy pp. 297-304 ACM 2015.

  • [43] S. Chen R. Wang X. Wang and K. Zhang “Side-channel leaks in web applications: A reality today a challenge tomorrow” in Security and Privacy (SP) 2010 IEEE Symposium on pp. 191-206 IEEE 2010.

  • [44] S. Khattak L. Simon and S. J. Murdoch “Systemization of pluggable transports for censorship resistance” arXiv preprint arXiv:1412.7448 2014.

  • [45] C. V. Wright L. Ballard S. E. Coull F. Monrose and G. M. Masson “Spot me if you can: Uncovering spoken phrases in encrypted voip conversations” in Security and Privacy 2008. SP 2008. IEEE Symposium on pp. 35-49 IEEE 2008.

  • [46] A. M. White A. R. Matthews K. Z. Snow and F. Monrose “Phonotactic reconstruction of encrypted voip conversations: Hookt on fon-iks” in Security and Privacy (SP) 2011 IEEE Symposium on pp. 3-18 IEEE 2011.

  • [47] S. Jana and V. Shmatikov “Memento: Learning secrets from process footprints” in Security and Privacy (SP) 2012 IEEE Symposium on pp. 143-157 IEEE 2012.

Journal information
Cited By
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 562 321 16
PDF Downloads 230 125 8