To Permit or Not to Permit, That is the Usability Question: Crowdsourcing Mobile Apps’ Privacy Permission Settings

Abstract

Millions of apps available to smartphone owners request various permissions to resources on the devices including sensitive data such as location and contact information. Disabling permissions for sensitive resources could improve privacy but can also impact the usability of apps in ways users may not be able to predict. We study an efficient approach that ascertains the impact of disabling permissions on the usability of apps through large-scale, crowdsourced user testing with the ultimate goal of making recommendations to users about which permissions can be disabled for improved privacy without sacrificing usability.

We replicate and significantly extend previous analysis that showed the promise of a crowdsourcing approach where crowd workers test and report back on various configurations of an app. Through a large, between-subjects user experiment, our work provides insight into the impact of removing permissions within and across different apps (our participants tested three apps: Facebook Messenger (N=218), Instagram (N=227), and Twitter (N=110)). We study the impact of removing various permissions within and across apps, and we discover that it is possible to increase user privacy by disabling app permissions while also maintaining app usability.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] Yuvraj Agarwal and Malcolm Hall. 2013. ProtectMyPrivacy: detecting and mitigating privacy leaks on iOS devices using crowdsourcing. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. ACM, 97–110.

  • [2] Shahriyar Amini, Jialiu Lin, Jason I Hong, Janne Lindqvist, and Joy Zhang. 2013. Mobile Application Evaluation Using Automation and Crowdsourcing. In Workshop on Privacy Enhancing Tools.

  • [3] Karissa Bell. 2015. The 7 best iPhone photography apps of all time. (2015). http://mashable.com/2015/12/13/best-iphone-photo-apps-of-all-time/#NruakC5LTuqF.

  • [4] K. Benton, L. J. Camp, and V. Garg. 2013. Studying the effectiveness of Android application permissions requests. In Security and Social Networking (SESOC), 2013 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops). 291–296.

  • [5] Sven Bugiel, Lucas Davi, Alexandra Dmitrienko, Thomas Fischer, Ahmad-Reza Sadeghi, and Bhargava Shastry. 2012. Towards Taming Privilege-Escalation Attacks on Android. In 19th Network & Distributed System Security Symposium.

  • [6] Iker Burguera, Urko Zurutuza, and Simin Nadjm-Tehrani. 2011. Crowdroid: behavior-based malware detection system for android. In Proceedings of the 1st ACM workshop on Security and privacy in smartphones and mobile devices. ACM, 15–26.

  • [7] L. Jean Camp and Allan Friedman. 2004. Peer Patching – Rapid Response in Distributed Systems. In 24th Army Science Conference.

  • [8] Scott Clifford, Ryan M Jewell, and Philip D Waggoner. 2015. Are samples drawn from Mechanical Turk valid for research on political ideology? Research & Politics 2, 4 (2015), 2053168015622072.

  • [9] Ryszard Wiśniewski Connor Tumbleson. 2015. A tool for reverse engineering Android apk files. (2015). https://ibotpeaches.github.io/Apktool/.

  • [10] E. Damiani, S. De Capitani Di Vimercati, S. Paraboschi, and P. Samarati. 2004. P2P-Based Collaborative Spam Detection and Filtering. In Fourth IEEE Conference on P2P. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1334945

  • [11] Developers. 2016. Requesting Permissions. (2016). https://developer.android.com/guide/topics/permissions/requesting.html.

  • [12] Irit Dinur and Kobbi Nissim. 2003. Revealing information while preserving privacy. In Proceedings of the twenty-second ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems. ACM, 202–210.

  • [13] Alex Dobie. 2015. The best photography apps for Android. (2015). http://www.androidcentral.com/best-photography-apps-android.

  • [14] Zheng Dong and L Jean Camp. 2012. PeerSec: Towards Peer Production and Crowdsourcing for Enhanced Security.. In HotSec.

  • [15] Zheng Dong, Vaibhav Garg, Jean Camp, and Apu Kapadia. 2012. Pools, Clubs and Security: Designing for a Party Not a Person. In Proceedings of The New Security Paradigms Workshop (NSPW). 77–86. DOI:http://dx.doi.org/10.1145/2413296.2413304

  • [16] Paul D. Ellis. 2009. Thresholds for Interpreting Effect Sizes. (2009). http://www.polyu.edu.hk/mm/effectsizefaqs/thresholds_for_interpreting_effect_sizes2.html.

  • [17] Kassem Fawaz and Kang G Shin. 2014. Location privacy protection for smartphone users. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. ACM, 239–250.

  • [18] Adrienne Porter Felt, Erika Chin, Steve Hanna, Dawn Song, and David Wagner. 2011a. Android permissions demystified. In 18th ACM Conference on Computer and Communications Security. 627–638. DOI:http://dx.doi.org/10.1145/2046707.2046779

  • [19] Adrienne Porter Felt, Kate Greenwood, and David Wagner. 2011b. The effectiveness of application permissions. In 2nd USENIX Conference on Web Application Development. 75–86.

  • [20] Adrienne Porter Felt, Elizabeth Ha, Serge Egelman, Ariel Haney, Erika Chin, and David Wagner. 2012. Android Permissions: User Attention, Comprehension, and Behavior. In 8th Symposium on Usable Privacy and Security. Article 3, 14 pages. DOI:http://dx.doi.org/10.1145/2335356.2335360

  • [21] Vaibhav Garg, Sameer Patil, Apu Kapadia, and L. Jean Camp. 2013. Peer-produced Privacy Protection. In IEEE International Symposium on Technology and Society (ISTAS). 147–154. DOI: http://dx.doi.org/10.1109/ISTAS.2013.6613114

  • [22] Hamza Harkous, Rameez Rahman, Bojan Karlas, and Karl Aberer. 2016. The Curious Case of the PDF Converter that Likes Mozart: Dissecting and Mitigating the Privacy Risk of Personal Cloud Apps. arXiv preprint arXiv:1608.05661 (2016).

  • [23] Jeff Howe. 2006. Crowdsourcing: A Definition. (2006). http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html.

  • [24] Qatrunnada Ismail, Tousif Ahmed, Apu Kapadia, and Michael K Reiter. 2015. Crowdsourced exploration of security configurations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 467–476.

  • [25] ISO. 2016. Usability of consumer products and products for public use. (2016). https://www.iso.org/obp/ui/#iso:std:iso:ts:20282:-2:ed-2:v1:en.

  • [26] Ruogu Kang, Stephanie Brown, Laura Dabbish, and Sara Kiesler. 2014. Privacy Attitudes of Mechanical Turk Workers and the U.S. Public. In Symposium On Usable Privacy and Security (SOUPS ’14). 37–49.

  • [27] Patrick Gage Kelley, Sunny Consolvo, Lorrie Faith Cranor, Jaeyeon Jung, Norman Sadeh, and David Wetherall. 2012. A conundrum of permissions: Installing applications on an Android smartphone. In Financial Cryptography and Data Security. Springer, 68–79.

  • [28] Patrick Gage Kelley, Lorrie Faith Cranor, and Norman Sadeh. 2013. Privacy as part of the app decision-making process. In 2013 ACM Conference on Human Factors in Computing Systems. 3393–3402.

  • [29] Kristen Kennedy, Eric Gustafson, and Hao Chen. 2013. Quantifying the Effects of Removing Permissions from Android Applications. In IEEE Mobile Security Technologies.

  • [30] Kim Komando. 2015. These 7 apps are among the worst at protecting privacy. (2015). http://www.usatoday.com/story/tech/columnist/komando/2015/09/18/apps-protecting-privacy/32563419/.

  • [31] Robert Kosara and Caroline Ziemkiewicz. 2010. Do Mechanical Turks dream of square pie charts?. In Proceedings of the 3rd BELIV’10 Workshop: Beyond time and errors: novel evaluation methods for Information Visualization. ACM, 63–70.

  • [32] Jialiu Lin, Bin Liu, Norman Sadeh, and Jason I Hong. 2014. Modeling users’ mobile app privacy preferences: Restoring usability in a sea of permission settings. In Symposium On Usable Privacy and Security (SOUPS 2014). 199–212.

  • [33] Jialiu Lin, Norman Sadeh, Shahriyar Amini, Janne Lindqvist, Jason I Hong, and Joy Zhang. 2012. Expectation and purpose: Understanding users’ mental models of mobile app privacy through crowdsourcing. In 2012 ACM Conference on Ubiquitous Computing. 501–510.

  • [34] Bin Liu, Mads Schaarup Andersen, Florian Schaub, Hazim Almuhimedi, Shikun Aerin Zhang, Norman Sadeh, Yuvraj Agarwal, and Alessandro Acquisti. 2016. Follow My Recommendations: A Personalized Assistant for Mobile App Permissions. In Twelfth Symposium on Usable Privacy and Security (SOUPS 2016).

  • [35] Di Liu, Randolph G. Bias, Matthew Lease, and Rebecca Kuipers. 2012. Crowdsourcing for usability testing. Proceedings of the American Society for Information Science and Technology 49, 1 (2012), 1–10. DOI:http://dx.doi.org/10.1002/meet.14504901100

  • [36] Craig M. MacDonald, Sean Fitzell, Megan Koontz, Kate Merlie, Alana Miller, Samantha Raddatz, Tal Rozen, April Siqueiros, and Susan Young. 2013. Usability Report. (2013). http://www.craigmacdonald.com/wp-content/uploads/2013/08/CCPS-Usability-Report-Summer-2013.pdf.

  • [37] Michelle L Mazurek, Saranga Komanduri, Timothy Vidas, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Patrick Gage Kelley, Richard Shay, and Blase Ur. 2013. Measuring password guessability for an entire university. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security. ACM, 173–186.

  • [38] Eyal Peer, Joachim Vosgerau, and Alessandro Acquisti. 2014. Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods 46, 4 (2014), 1023–1031.

  • [39] PrivacyGrade. 2014. PrivacyGrade: Grading The Privacy Of Smartphone Apps. (2014). http://www.privacygrade.org/.

  • [40] John Patrick Pullen. 2015. Your Favorite Apps Know More About You Than You Realize. (2015). http://time.com/3857380/apps-security-privacy-trivia-crack/.

  • [41] Bahman Rashidi, Carol Fung, and Tam Vu. 2015. Dude, ask the experts!: Android resource access permission recommendation with RecDroid. In Integrated Network Management (IM), 2015 IFIP/IEEE International Symposium on. 296–304. DOI:http://dx.doi.org/10.1109/INM.2015.7140304

  • [42] M. K. Reiter and S. G. Stubblebine. 1998. Resilient authentication using path independence. IEEE Trans. Comput. 47, 12 (1998), 1351–1362.

  • [43] Meredith Rizzo. 2014. How Well Do Your Apps Protect Your Privacy? (2014). http://www.npr.org/sections/health-shots/2014/11/20/363342736/how-well-do-your-apps-protect-your-privacy.

  • [44] Franziska Roesner, Tadayoshi Kohno, Alexander Moshchuk, Bryan Parno, Helen J Wang, and Crispin Cowan. 2012. Userdriven access control: Rethinking permission granting in modern operating systems. In Security and privacy (SP), 2012 IEEE Symposium on. IEEE, 224–238.

  • [45] Joel Ross, Lilly Irani, M. Six Silberman, Andrew Zaldivar, and Bill Tomlinson. 2010. Who Are the Crowdworkers?: Shifting Demographics in Mechanical Turk. In CHI ’10 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’10). ACM, New York, NY, USA, 2863–2872.

  • [46] Jeff Sauro. 2010. If You Could Only Ask One Question, Use This One. (2010). http://www.measuringu.com/blog/single-question.php.

  • [47] Jeff Sauro. 2011. Measuring Usability With The System Usability Scale (SUS). (2011). http://www.measuringu.com/sus.php.

  • [48] Jeff Sauro. 2012. 10 Things To Know About The Single Ease Question (SEQ). (2012). http://www.measuringu.com/blog/seq10.php.

  • [49] Jeff Sauro and Joseph S Dumas. 2009. Comparison of three one-question, post-task usability questionnaires. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1599–1608.

  • [50] Umesh Shankar and Chris Karlof. 2006. Doppelganger: Better browser privacy without the bother. In Proceedings of the 13th ACM conference on Computer and communications security. ACM, 154–167.

  • [51] Statista. 2016. Most popular global mobile messenger apps as of January 2016, based on number of monthly active users (in millions). (2016). http://www.statista.com/statistics/258749/most-popular-global-mobile-messenger-apps/.

  • [52] Statista. 2017a. Number of monthly active Facebook Messenger users from April 2014 to April 2017 (in millions). (2017). https://www.statista.com/statistics/417295/facebook-messenger-monthly-active-users/.

  • [53] Statista. 2017b. Number of monthly active Instagram users from January 2013 to April 2017 (in millions). (2017). https://www.statista.com/statistics/253577/number-of-monthly-active-instagram-users/.

  • [54] Statista. 2017c. Number of monthly active Twitter users worldwide from 1st quarter 2010 to 1st quarter 2017 (in millions). (2017). https://www.statista.com/statistics/282087/number-of-monthly-active-twitter-users/.

  • [55] Eran Toch. 2014. Crowdsourcing privacy preferences in context-aware applications. Personal and ubiquitous computing 18, 1 (2014), 129–141.

  • [56] Voidcan. 2015. Top 10 Microblogging Sites for 2015. (2015). http://www.voidcan.org/top-10-microblogging-sites-for-2015/.

  • [57] Haoyu Wang, Jason Hong, and Yao Guo. 2015. Using text mining to infer the purpose of permission use in mobile apps. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1107–1118.

  • [58] D. Wendlandt, D. G. Andersen, and A. Perrig. 2008. Perspectives: Improving SSH-style host authentication with multi-path probing. In Proceedings of USENIX Annual Technical Conference.

  • [59] Primal Wijesekera, Arjun Baokar, Ashkan Hosseini, Serge Egelman, David Wagner, and Konstantin Beznosov. 2015. Android permissions remystified: a field study on contextual integrity. In 24th USENIX Security Symposium (USENIX Security 15). 499–514.

  • [60] Wiki. 2016. Fair Payment. (2016). http://wiki.wearedynamo.org/index.php?title=Fair_payment.

  • [61] Rebecca N Wright, L Jean Camp, Ian Goldberg, Ronald L Rivest, and Graham Wood. 2002. Privacy tradeoffs: myth or reality?. In Financial Cryptography. Springer, 147–151.

  • [62] M. Yuen, I. King, and K. Leung. 2011. A Survey of Crowdsourcing Systems. In in Proceedings the 3rd SocialCom. http://www.cse.cuhk.edu.hk/~king/PUB/SocialCom2011-Yuen.pdf

OPEN ACCESS

Journal + Issues

Search