Portrait of a Privacy Invasion

Detecting Relationships Through Large-scale Photo Analysis

Open access


The popularity of online social networks has changed the way in which we share personal thoughts, political views, and pictures. Pictures have a particularly important role in the privacy of users, as they can convey substantial information (e.g., a person was attending an event, or has met with another person). Moreover, because of the nature of social networks, it has become increasingly difficult to control who has access to which content. Therefore, when a substantial amount of pictures are accessible to one party, there is a very serious potential for violations of the privacy of users. In this paper, we demonstrate a novel technique that, given a large corpus of pictures shared on a social network, automatically determines who is dating whom, with reasonable precision. More specifically, our approach combines facial recognition, spatial analysis, and machine learning techniques to determine pairs that are dating. To the best of our knowledge, this is the first privacy attack of this kind performed on social networks. We implemented our approach in a tool, called Creepic, and evaluated it on two real-world datasets. The results show that it is possible to automatically extract non-obvious, and nondisclosed, relationships between people represented in a group of pictures, even when the people involved are not directly part of a connected social clique.

[1] A. Acquisti, R. Gross, and F. Stutzman. Faces of facebook: Privacy in the age of augmented reality. 2011.

[2] APeWatch. App Permission Watcher. http://apewatch.de/.

[3] M. Balduzzi, C. Platzer, T. Holz, E. Kirda, D. Balzarotti, and C. Kruegel. Abusing social networks for automated user profiling. In Recent Advances in Intrusion Detection, pages 422-441. Springer, 2010.

[4] M. Bar-Hillel. The base-rate fallacy in probability judgments. Acta Psychologica, 44(3):211-233, 1980.

[5] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 19(7):711-720, 1997.

[6] C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

[7] D. J. Crandall, L. Backstrom, D. Cosley, S. Suri, D. Huttenlocher, and J. Kleinberg. Inferring social ties from geographic coincidences. Proceedings of the National Academy of Sciences, 107(52):22436-22441, 2010.

[8] C. A. Davis Jr, G. L. Pappa, D. R. R. de Oliveira, and F. de L Arcanjo. Inferring the location of twitter messages based on user relationships. Transactions in GIS, 15(6):735-751, 2011.

[9] Facebook. The Formulation of Love. https: //www.facebook.com/notes/facebook-data-science/theformation- of-love/10152064609253859.

[10] Facebook, Ericcson, and Qualcomm. A Focus on Efficiency. https://fbcdn-dragon-a.akamaihd.net/hphotos-akash3/ 851590_229753833859617_1129962605_n.pdf.

[11] GigaOM. Instagram reports 200M users. http://gigaom.com/2014/03/25/instagramreports-200m-users-20b-photos-shared/.

[12] P. Gundecha, G. Barbier, and H. Liu. Exploiting vulnerability to secure user privacy on a social networking site. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 511-519. ACM, 2011.

[13] T. Independent. How average Briton is caught on camera 300 times a day. http://www.independent.co.uk/news/uk/thisbritain/ how-average-briton-is\discretionary{-}{}{}caught-oncamera-300-times-a-day-572781.html.

[14] C. Jernigan and B. F. Mistree. Gaydar: Facebook friendships expose sexual orientation. First Monday, 14(10), 2009.

[15] J. K. Lars Backstrom. Romantic partnerships and the dispesion of social ties: A network analysis of relationship status on facebook. In Proceedings of the 2014 ACM conference on Computer supported cooperative work. ACM, 2014.

[16] B. W. Matthews. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2):442-451, 1975.

[17] S. Nagaraja, P. Schaffer, and D. Aouada. Who clicks there!: anonymising the photographer in a camera saturated society. In Proceedings of the 10th annual ACM workshop on Privacy in the electronic society, pages 13-22. ACM, 2011.

[18] OpenCV. OpenCV. http://opencv.org.

[19] E. Osuna, R. Freund, and F. Girosit. Training support vector machines: an application to face detection. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on, pages 130-136. IEEE, 1997.

[20] B. Rouberol. Comparison of face detection tools. http://bit.ly/1olWs4n.

[21] H. A. Rowley, S. Baluja, and T. Kanade. Neural networkbased face detection. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 20(1):23-38, 1998.

[22] M. Turk and A. Pentland. Eigenfaces for recognition. Journal of cognitive neuroscience, 3(1):71-86, 1991.

[23] T. Verge. The new Snapchat brilliantly mixes video and texting. http://www.theverge.com/2014/5/1/ 5670260/real-talk-the-new-snapchatmakes- texting-fun-again-video-calls.

[24] P. Wagner. Facerec. https://github.com/bytefish/facerec.

[25] P. I. Wilson and J. Fernandez. Facial feature detection using haar classifiers. Journal of Computing Sciences in Colleges, 21(4):127-133, 2006.

Journal Information


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 60 60 28
PDF Downloads 12 12 6