Recent studies have shown that machine learning can identify individuals with mental illnesses by analyzing their social media posts. Topics and words related to mental health are some of the top predictors. These findings have implications for early detection of mental illnesses. However, they also raise numerous privacy concerns. To fully evaluate the implications for privacy, we analyze the performance of different machine learning models in the absence of tweets that talk about mental illnesses. Our results show that machine learning can be used to make predictions even if the users do not actively talk about their mental illness. To fully understand the implications of these findings, we analyze the features that make these predictions possible. We analyze bag-of-words, word clusters, part of speech n-gram features, and topic models to understand the machine learning model and to discover language patterns that differentiate individuals with mental illnesses from a control group. This analysis confirmed some of the known language patterns and uncovered several new patterns. We then discuss the possible applications of machine learning to identify mental illnesses, the feasibility of such applications, associated privacy implications, and analyze the feasibility of potential mitigations.
 Mohammed Al-Mosaiwi and Tom Johnstone. In an absolute state: elevated use of absolutist words is a marker specific to anxiety depression and suicidal ideation. Clinical Psychological Science January 2018.
 Nazanin Andalibi Pinar Ozturk and Andrea Forte. Sensitive Self-disclosures Responses and Social Support on Instagram: The Case of #Depression. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing CSCW ’17 pages 1485–1500 New York NY USA 2017. ACM.
 Athanasios Andreou Marcio Silva Fabrício Benevenuto Oana Goga Patrick Loiseau and Alan Mislove. Measuring the Facebook Advertising Ecosystem. In NDSS 2019 - Proceedings of the Network and Distributed System Security Symposium San Diego United States February 2019.
 Stefan Axelsson. The Base-rate Fallacy and Its Implications for the Difficulty of Intrusion Detection. In Proceedings of the 6th ACM Conference on Computer and Communications Security CCS ’99 pages 1–7 New York NY USA 1999. ACM.
 Steven Bird Ewan Klein and Edward Loper. Natural Language Processing with Python. O’Reilly Media Inc. 1st edition 2009.
 Peter F. Brown Peter V. deSouza Robert L. Mercer Vincent J. Della Pietra and Jenifer C. Lai. Class-based N-gram Models of Natural Language. Comput. Linguist. 18(4):467–479 December 1992.
 Glen Coppersmith Mark Dredze and Craig Harman. Quantifying Mental Health Signals in Twitter. Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality pages 51–60 2014.
 Glen Coppersmith Mark Dredze Craig Harman and Kristy Hollingshead. From ADHD to SAD: Analyzing the Language of Mental Health on Twitter through Self-Reported Diagnoses. Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality pages 1–10 2015.
 Glen Coppersmith Mark Dredze Craig Harman Kristy Hollingshead and Margaret Mitchell. CLPsych 2015 Shared Task: Depression and PTSD on Twitter. the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality pages 31–39 2015.
 Glen Coppersmith Craig Harman and Mark Dredze. Measuring Post Traumatic Stress Disorder in Twitter. In Proceedings of the 7th International AAAI Conference on Weblogs and Social Media (ICWSM). 2(1):23–45 2014.
 Munmun De Choudhury Michael Gamon Scott Counts and Eric Horvitz. Predicting Depression via Social Media. In Proceedings of the 7th International AAAI Conference on Weblogs and Social Media pages 128–138 2013.
 Olive Jean Dunn. Multiple comparisons among means. Journal of the American Statistical Association 56(293):52–64 1961.
 Kevin Gimpel Nathan Schneider Brendan O’Connor Dipanjan Das Daniel Mills Jacob Eisenstein Michael Heilman Dani Yogatama Jeffrey Flanigan and Noah A. Smith. Part-of-speech Tagging for Twitter: Annotation Features and Experiments. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2 HLT ’11 pages 42–47 Stroudsburg PA USA 2011. Association for Computational Linguistics.
 Su Golder Shahd Ahmed Gill Norman and Andrew Booth. Attitudes toward the ethics of research using social media: A systematic review June 2017.
 Sharath Chandra Guntuku David B. Yaden Margaret L. Kern Lyle H. Ungar and Johannes C. Eichstaedt. Detecting depression and mental illness on social media: an integrative review 2017.
 Seda Gurses Rebekah Overdorf and Ero Balsa. POTs: The revolution will not be optimized? 11th Hot Topics in Privacy Enhancing Technologies (HotPETs) 2018.
 Harward Harvard Medical School. National Comorbidity Survey (NCS). https://www.hcp.med.harvard.edu/ncs/index.php 2007. [Online; Accessed 26-April-2018 Ref Data Table 2: 12-month prevalence DSM-IV/WMH-CIDI disorders by sex and cohort (https://www.hcp.med.harvard.edu/ncs/ftpdir/table_ncsr_12monthprevgenderxage.pdf)].
 Melinda R Hess and Jeffrey D Kromrey. Robust confidence intervals for effect sizes: A comparative study of cohen’sd and cliff’s delta under non-normality and heterogeneous variances. 2004.
 Alvarez-Conrad Jennifer Zoellner Lori A. and Foa Edna B. Linguistic predictors of trauma pathology and physical health. Applied Cognitive Psychology 15(7):S159–S170 2001.
 Ron Kohavi. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence - Volume 2 IJCAI’95 pages 1137–1143 San Francisco CA USA 1995. Morgan Kaufmann Publishers Inc.
 Björn Lantz. The large sample size fallacy. Scandinavian journal of caring sciences 27(2):487–492 2013.
 Naomi Lee. Trouble on the radar. The Lancet 384(29):1917 2014.
 Huina Mao Xin Shuai and Apu Kapadia. Loose Tweets: An Analysis of Privacy Leaks on Twitter. In Proceedings of the 10th Annual ACM Workshop on Privacy in the Electronic Society WPES ’11 pages 1–12 New York NY USA 2011. ACM.
 Jon D Mcauliffe and David M Blei. Supervised topic models. In Advances in neural information processing systems pages 121–128 2008.
 Jude Mikal Samantha Hurst and Mike Conway. Ethical issues in using Twitter for population-level depression monitoring: A qualitative study. BMC Medical Ethics 17(1) 2016.
 Nilly Mor and Jennifer Winquist. Self-Focused Attention and Negative Affect : A Meta-Analysis. Psychological bulletin 128(4):638–662 2002.
 Olutobi Owoputi Brendan O’Connor Chris Dyer Kevin Gimpel and Nathan Schneider. Part-of-speech tagging for twitter: Word clusters and other advances. 2012.
 Nicolas Papernot Patrick McDaniel and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 2016.
 James W Pennebaker and Laura A King. Linguistic styles: Language use as an individual difference. Journal of personality and social psychology 77(6):1296 1999.
 Daniel Preoţiuc-Pietro Johannes Eichstaedt Gregory Park Maarten Sap Laura Smith Victoria Tobolsky H Andrew Schwartz and Lyle Ungar. The role of personality age and gender in tweeting about mental illness. In Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality pages 21–30 2015.
 Philip Resnik William Armstrong Leonardo Claudino Thang Nguyen Viet-An Nguyen and Jordan Boyd-graber. Beyond LDA : Exploring Supervised Topic Modeling for Depression-Related Language in Twitter. Proceedings of the 52nd Workshop Computational Linguistics and Clinical Psychology 1(2014):99–107 2015.
 Philip Resnik William Armstrong Leonardo Claudino Thang Nguyen Viet-An Nguyen and Jordan Boyd-Graber. The University of Maryland CLPsych 2015 Shared Task System. Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality pages 54–60 2015.
 Philip Resnik Anderson Garron and Rebecca Resnik. Using Topic Modeling to Improve Prediction of Neuroticism and Depression. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) pages 1348–1353 2013.
 Guillaume A. Rousselet. Cohen’s d is biased. https://garstats.wordpress.com/2018/04/04/dbias/ 2018. [Online; Accessed 26-November-2018].
 Beatrice Santorini. Part-of-speech tagging guidelines for the Penn Treebank Project (3rd revision). Technical Reports (CIS) page 570 1990.
 H Andrew Schwartz Johannes C Eichstaedt Margaret L Kern Lukasz Dziurzynski Stephanie M Ramones Megha Agrawal Achal Shah Michal Kosinski David Stillwell Martin E.P. Seligman and Lyle H Ungar. Personality Gender and Age in the Language of Social Media: The Open-Vocabulary Approach. PLoS ONE 8(9) 2013.
 Manya Sleeper Justin Cranshaw Patrick Gage Kelley Blase Ur Alessandro Acquisti Lorrie Faith Cranor and Norman Sadeh. “I read my Twitter the next morning and was astonished” a conversational perspective on Twitter regrets. Chi pages 3277–3286 2013.
 Daria Smirnova Paul Cumming Elena Sloeva Natalia Kuvshinova Dmitry Romanov and Gennadii Nosachev. Language patterns discriminate mild depression from normal sadness and euthymic state. Frontiers in Psychiatry 9:105 2018.
 Gail M Sullivan and Richard Feinn. Using effect size—or why the p value is not enough. Journal of graduate medical education 4(3):279–282 2012.
 Yla R Tausczik and James W Pennebaker. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology 29(1):24–54 2010.
 The Federal Trade Comission. Data brokers: A call for transparency and accountability. Data Brokers and the Need for Transparency and Accountability pages 1–101 2014.
 Rianne Van der Zanden Keshia Curie Monique Van Londen Jeannet Kramer Gerard Steen and Pim Cuijpers. Web-based depression treatment: Associations of clients’ word use with adherence and outcome. Journal of Affective Disorders 160:10–13 2014.
 Yang Wang Gregory Norcie Saranga Komanduri Alessandro Acquisti Pedro Giovanni Leon and Lorrie Faith Cranor. “I Regretted the Minute I Pressed Share”: A Qualitative Study of Regrets on Facebook. In Proceedings of the Seventh Symposium on Usable Privacy and Security SOUPS ’11 pages 10:1–10:16 New York NY USA 2011. ACM.
 Johannes Zimmermann Timo Brockmeyer Matthias Hunn Henning Schauenburg and Markus Wolf. First-person Pronoun Use in Spoken Language as a Predictor of Future Depressive Symptoms: Preliminary Evidence from a Clinical Sample of Depressed Patients. Clinical Psychology & Psychotherapy 24(2):384–391 mar 2017.