Open Access

Evaluating stance-annotated sentences from the Brexit Blog Corpus: A quantitative linguistic analysis


Cite

Adar, Eytan, Li Zhang, Lada A. Adamic and Rajan M. Lukose. 2004. Implicit structure and the dynamics of blogspace. Workshop on the Weblogging Ecosystem 13 (1): 16989–16995.Search in Google Scholar

Agarwal, Nitin and Huan Liu. 2008. Blogosphere: Research issues, tools, and applications. ACM SIGKDD Explorations Newsletter 10 (1): 18–31.10.1145/1412734.1412737Search in Google Scholar

Anand, Pranav, Marilyn Walker, Rob Abbott, Jean E. Fox Tree, Robeson Bowmani and Michael Minor. 2011. Cats rule and dogs drool!: Classifying stance in online debate. In Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis, WASSA ’11, 1–9. Stroudsburg, PA, USA: Association for Computational Linguistics.Search in Google Scholar

Bassiouney, Reem. 2012. Politicizing identity: Code choice and stance-taking during the Egyptian revolution. Discourse & Society 23 (2): 107–126.10.1177/0957926511431514Search in Google Scholar

Benveniste, Émile. 1971. Subjectivity in language. In M. E. Meek (ed.). Problems in general linguistics, 223–230. Coral Gables, FL: University of Miami Press.Search in Google Scholar

Berman, Ruth, Hrafnhildur Ragnarsdóttir and Sven Strömqvist. 2002. Discourse stance: Written and spoken language. Written Language & Literacy 5 (2): 253–287.10.1075/wll.5.2.06berSearch in Google Scholar

Biber, Douglas. 2006. Stance in spoken and written university registers. Journal of English for Academic Purposes 5 (2): 97–116.10.1016/j.jeap.2006.05.001Search in Google Scholar

Cabrejas-Peñuelas, Ana B. and Mercedes Díez-Prados. 2014. Positive self-evaluation versus negative other-evaluation in the political genre of pre-election debates. Discourse & Society 25 (2): 159–185.10.1177/0957926513515601Search in Google Scholar

Cataldi, Cataldi, Mario, Luigi Di Caro and Claudio Schifanella. 2010. Emerging topic detection on twitter based on temporal and social terms evaluation. In Proceedings of the Tenth International Workshop on Multimedia Data Mining 4, 1–10. Washington, DC, USA: Association for Computing Machinery.10.1145/1814245.1814249Search in Google Scholar

Chaemsaithong, Krisda. 2012. Performing self on the witness stand: Stance and relational work in expert witness testimony. Discourse & Society 23 (5): 465–486.10.1177/0957926512441111Search in Google Scholar

Chiluwa, Innocent and Presley Ifukor. 2015. ‘War against our Children’: Stance and evaluation in #BringBackOurGirls campaign discourse on Twitter and Facebook. Discourse & Society 26 (3): 267–296.10.1177/0957926514564735Search in Google Scholar

Conrad, Susan and Douglas Biber. 2000. Adverbial marking of stance in speech and writing. In G. Thompson (ed.). Evaluation in text: Authorial stance and the construction of discourse, 56–73. Oxford: Oxford University Press.Search in Google Scholar

Downing, Angela. 2001. “Surely you knew!”: Surely as a marker of evidentiality and stance. Functions of Language 8 (2): 251–282.10.1075/fol.8.2.05dowSearch in Google Scholar

Du Bois, John. 2007. The stance triangle. In R. Englebretson (ed.). Stancetaking in discourse: Subjectivity, evaluation, interaction, 139–182. Amsterdam: John Benjamins.10.1075/pbns.164.07duSearch in Google Scholar

Ekberg, Lena and Carita Paradis. 2009. Editorial: Evidentiality in language and cognition. Functions of Language 16 (1): 5–7.10.1075/fol.16.1.02ekbSearch in Google Scholar

Englebretson, Robert. 2007. Stancetaking in discourse: An introduction. In R. Englebretson (ed.). Stancetaking in discourse: Subjectivity, evaluation, interaction, 1–25. Amsterdam: John Benjamins.10.1075/pbns.164.02engSearch in Google Scholar

Facchinetti, Roberta, Frank Palmer and Manfred Krug (eds.). 2003. Modality in contemporary English (Topics in English Linguistics 44). Berlin: Walter de Gruyter.10.1515/9783110895339Search in Google Scholar

Faulkner, Adam. 2014. Automated classification of stance in student essays: An approach using stance target information and the Wikipedia link-based measure. Science 376 (12): 86.Search in Google Scholar

Ferreira, William and Andreas Vlachos. 2016. Emergent: A novel data-set for stance classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1163–1168. Sheffield, UK.Search in Google Scholar

Fuoli, Matteo. 2012. Assessing social responsibility: A quantitative analysis of Appraisal in BP’s and IKEA’s social reports. Discourse & Communication 6 (1): 55–81.10.1177/1750481311427788Search in Google Scholar

Glynn, Dylan and Mette Sjölin. 2015. Subjectivity and epistemicity: Corpus, discourse, and literary approaches to stance. In D. Glynn and M. Sjölin (eds.). Corpus, discourse, and literary approaches to stance (Lund Studies in English 117), 360–410. Lund: Lund University.Search in Google Scholar

Granger, Sylviane. 2003. The international corpus of learner English: A new resource for foreign language learning and teaching and second language acquisition research. Tesol Quarterly 37 (3): 538–546.10.2307/3588404Open DOISearch in Google Scholar

Gray, Bethany and Douglas Biber. 2014. Stance markers. In K. Aijmer and C. Rühlemann (eds.). Corpus pragmatics: A handbook, 219–248. Cambridge: Cambridge University Press.10.1017/CBO9781139057493.012Search in Google Scholar

Gu, Xiang. 2015. Evidentiality, subjectivity and ideology in the Japanese history textbook. Discourse & Society 26 (1): 29–51.10.1177/0957926514543225Search in Google Scholar

Hasan, Kazi Saidul and Vincent Ng. 2013a. Stance classification of ideological debates: Data, models, features, and constraints. In Proceeding of IJCNLP 2013: The 6th International Joint Conference on Natural Language Processing, 1348–1356. Nagoya, Japan.Search in Google Scholar

Hasan, Kazi Saidul and Vincent Ng. 2013b. Frame semantics for stance classification. In Proceedings of CoNLL 2013: The Seventeenth Conference on Computational Natural Language Learning, 124–132. Sofia, Bulgaria.Search in Google Scholar

Hasan, Kazi Saidul and Vincent Ng. 2013c. Extra-linguistic constraints on stance recognition in ideological debates. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Vol. 2: Short Papers), 816–821. Sofia, Bulgaria.Search in Google Scholar

Hasan, Kazi Saidul and Vincent Ng. 2014. Why are you taking this stance? Identifying and classifying reasons in ideological debates. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 751–762. Doha, Qatar.10.3115/v1/D14-1083Search in Google Scholar

Hunston, Susan and Geoffrey Thompson (eds.). 2000. Evaluation in text: Authorial stance and the construction of discourse. Oxford: Oxford University Press.Search in Google Scholar

Hyland, Ken. 2005. Stance and engagement: A model of interaction in academic discourse. Discourse Studies 7 (2): 173–192.10.1177/1461445605050365Open DOISearch in Google Scholar

Jiang, Feng Kevin. 2017. Stance and voice in academic writing. International Journal of Corpus Linguistics 22 (1): 85–106.10.1075/ijcl.22.1.04jiaSearch in Google Scholar

Kanté, Issa. 2010. Mood and modality in finite noun complement clauses: A French-English contrastive study. International Journal of Corpus Linguistics 15 (2): 267–290.10.1075/ijcl.15.2.06kanSearch in Google Scholar

Kärkkäinen, Elise. 2003. Epistemic stance in English conversation: A description of its interactional functions, with a focus on I think (Pragmatics & Beyond New Series 115). Amsterdam: John Benjamins.10.1075/pbns.115Search in Google Scholar

Kessler, Brett, Geoffrey Numberg and Hinrich Schütze. 1997. Automatic detection of text genre. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics, 32–38. Association for Computational Linguistics.10.3115/976909.979622Search in Google Scholar

Kucher, Kostiantyn, Andreas Kerren, Carita Paradis and Magnus Sahlgren. 2016a. Visual analysis of text annotations for stance classification with ALVA. In EuroVis 2016: The 18th EG/VGTC Conference on Visualization, 49–51. Eurographics – European Association for Computer Graphics.Search in Google Scholar

Kucher, Kostiantyn, Teri Schamp-Bjerede, Andreas Kerren, Carita Paradis and Magnus Sahlgren. 2016b. Visual analysis of online social media to open up the investigation of stance phenomena. Information Visualization 15 (2): 93–116.10.1177/1473871615575079Search in Google Scholar

Kucher, Kostiantyn, Carita Paradis, Magnus Sahlgren and Andreas Kerren. 2017. Active learning and visual analytics for stance classification with ALVA. ACM Transactions on Interactive Intelligent Systems (TiiS) 7 (3): 1–31.10.1145/3132169Search in Google Scholar

Martin, James R. and Peter R. White. 2003. The language of evaluation. London: Palgrave Macmillan.Search in Google Scholar

Mathioudakis, Michael and Nick Koudas. 2010. Twittermonitor: Trend detection over the twitter stream. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, 1155–1158. Association for Computing Machinery.10.1145/1807167.1807306Search in Google Scholar

Mohammad, Saif M., Parinaz Sobhani and Svetlana Kiritchenko. 2016. Stance and sentiment in tweets. arXiv preprint arXiv:1605.01655.Search in Google Scholar

Mukherjee, Arjun and Bing Liu. 2010. Improving gender classification of blog authors. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, 207–217. Association for Computational Linguistics.Search in Google Scholar

Nguyen, Dong, Rilana Gravel, Dolf Trieschnigg and Theo Meder. 2013. “How old do you think I am?” A study of language and age in Twitter. In Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media, 439–448. Cambridge, Massachusetts, USA.10.1609/icwsm.v7i1.14381Search in Google Scholar

Pak, Alexander and Patrick Paroubek. 2010. Twitter as a corpus for sentiment analysis and opinion mining. In Proceedings of The Seventh International Conference on Language Resources and Evaluation (LREC) (Vol. 10), 1320–1326. Valletta, Malta.Search in Google Scholar

Pang, Bo and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2 (1–2): 1–135.10.1561/1500000011Open DOISearch in Google Scholar

Paradis, Carita. 2003. Between epistemic modality and degree: The case of really. In R. Facchinetti, F. Palmer and M. Krug (eds.). Modality in contemporary English (Topics in English Linguistics 44), 191–222. Berlin: DeGruyter.10.1515/9783110895339.191Search in Google Scholar

Park, Jaram, Young Min Baek and Meeyoung Cha. 2014. Cross-cultural comparison of nonverbal cues in emoticons on twitter: Evidence from big data analysis. Journal of Communication 64 (2): 333–354.10.1111/jcom.12086Open DOISearch in Google Scholar

Paterson, Laura L., Laura Coffey-Glover and David Peplow. 2016. Negotiating stance within discourses of class: Reactions to Benefits Street. Discourse & Society 27 (2): 195–214.10.1177/0957926515611558Search in Google Scholar

Peersman, Claudia, Walter Daelemans and Leona Van Vaerenbergh. 2011. Predicting age and gender in online social networks. In Proceedings of the Third International Workshop on Search and Mining User-Generated Contents, 37–44. Association for Computational Linguistics.10.1145/2065023.2065035Search in Google Scholar

Persing, Isaac and Vincent Ng, V. 2016. Modeling stance in student essays. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2174–2184. Association for Computational Linguistics.10.18653/v1/P16-1205Search in Google Scholar

Põldvere, Nele, Matteo Fuoli and Carita Paradis. 2016. A study of dialogic expansion and contraction in spoken discourse using corpus and experimental techniques. Corpora 11 (2): 191–225.10.3366/cor.2016.0092Search in Google Scholar

Precht, Kristen. 2003. Stance moods in spoken English: Evidentiality and aspect in British and American conversation. Text (Special issue: Negotiating Heteroglossia: Social Perspectives on Evaluation) 23 (2): 239–257.10.1515/text.2003.010Search in Google Scholar

Rajadesingan, Ashwin and Huan Liu. 2014. Identifying users with opposing opinions in Twitter debates. In International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction, 153–160. Berlin: Springer International Publishing.10.1007/978-3-319-05579-4_19Search in Google Scholar

Read, Jonathon and John Carroll. 2012. Annotating expressions of appraisal in English. Language Resources and Evaluation 46 (3): 421–447.10.1007/s10579-010-9135-7Open DOISearch in Google Scholar

Saurí, Roser and James Pustejovsky. 2009. FactBank: A corpus annotated with event factuality. Language Resources and Evaluation 43 (3): 227–268.10.1007/s10579-009-9089-9Open DOISearch in Google Scholar

Scheffé, Henry. 1999 [1959]. The analysis of variance. New York City: John Wiley & Sons.Search in Google Scholar

Schwartz, Andrew, Johannes Eichstaedt, Margaret Kern, Lukasz Dziurzynski, Stephanie Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David Stillwell, Martin Seligman and Lyle Ungar. 2013. Personality, gender, and age in the language of social media: The open-vocabulary approach. PLOSONE 8 (9): e73791.Search in Google Scholar

Simaki, Vasiliki. 2015. Sociolinguistic research on web textual data (Doctoral dissertation, in Greek). University of Patras, Greece. Retrieved from: http://hdl.handle.net/10889/9422Search in Google Scholar

Simaki, Vasiliki, Christina Aravantinou, Iosif Mporas and Vasileios Megalooikonomou. 2015a. Using sociolinguistic inspired features for gender classification of web authors. In International Conference on Text, Speech, and Dialogue (TSD) (Lecture Notes in Computer Science, vol. 9302), 587–594. Berlin: Springer International Publishing.10.1007/978-3-319-24033-6_66Search in Google Scholar

Simaki, Vasiliki, Christina Aravantinou, Iosif Mporas and Vasileios Megalooikonomou. 2015b. Automatic estimation of web bloggers’ age using regression models. In International Conference on Speech and Computer (SPECOM), 113–120. Berlin: Springer International Publishing.10.1007/978-3-319-23132-7_14Search in Google Scholar

Simaki, Vasiliki, Christina Aravantinou, Iosif Mporas, Marianna Kondyli and Vasileios Megalooikonomou. 2017a. Sociolinguistic features for author gender identification: From qualitative evidence to quantitative analysis. Journal of Quantitative Linguistics 24 (1): 65–84.10.1080/09296174.2016.1226430Open DOISearch in Google Scholar

Simaki Vasiliki, Carita Paradis and Andreas Kerren. 2017b. Stance classification in texts from blogs on the 2016 British Referendum. In A. Karpov, R. Potapova and I. Mporas (eds.). Speech and computer. SPECOM 2017 (Lecture Notes in Computer Science, vol. 10458), 700–709. Berlin: Springer International Publishing.10.1007/978-3-319-66429-3_70Search in Google Scholar

Simaki, Vasiliki, Carita Paradis, Maria Skeppstedt, Magnus Sahlgren, Kostiantyn Kucher and Andreas Kerren. 2017c. Annotating speaker stance in discourse: The Brexit Blog Corpus. Corpus Linguistics and Linguistic Theory. DOI:10.1515/cllt-2016-006010.1515/cllt-2016-0060Open DOISearch in Google Scholar

Somasundaran, Swapna and Janyce Wiebe. 2010. Recognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, 116–124. Association for Computational Linguistics.Search in Google Scholar

Sridhar, Dhanya, Lise Getoor and Marilyn Walker. 2014. Collective stance classification of posts in online debate forums. In Proceedings of the Joint Workshop on Social Dynamics and Personal Attributes in Social Media, 109–117. Baltimore, Maryland, USA.10.3115/v1/W14-2715Search in Google Scholar

Stamatatos, Efstathios. 2009. A survey of modern authorship attribution methods. Journal of the American Society for Information Science and Technology 60 (3): 538–556.10.1002/asi.21001Open DOISearch in Google Scholar

Stamatatos, Efstathios, Nikos Fakotakis and George Kokkinakis. 2000. Automatic authorship attribution. In Proceedings of the Ninth Conference on European Chapter of the Association for Computational Linguistics, 158–164. Association for Computational Linguistics.10.3115/977035.977057Search in Google Scholar

Stamatatos, Efstathios, Nikos Fakotakis and George Kokkinakis. 2001. Computer-based authorship attribution without lexical measures. Computers and the Humanities 35 (2): 193–214.10.1023/A:1002681919510Open DOISearch in Google Scholar

Taboada, Maite. 2016. Sentiment analysis: An overview from linguistics. Annual Review of Linguistics 2: 325–347.10.1146/annurev-linguistics-011415-040518Open DOISearch in Google Scholar

Tracy, Karen. 2011. What’s in a name? Stance markers in oral argument about marriage laws. Discourse & Communication 5 (1): 65–88.10.1177/1750481310390167Search in Google Scholar

Tukey, John W. 1949. Comparing individual means in the analysis of variance. Biometrics 5 (2): 99–114.10.2307/3001913Open DOISearch in Google Scholar

Van de Kauter, Marjan, Bart Desmet and Véronique Hoste. 2015. The good, the bad and the implicit: A comprehensive approach to annotating explicit and implicit sentiment. Language Resources and Evaluation 49 (3): 685–720.10.1007/s10579-015-9297-4Search in Google Scholar

Verhagen, Arie. 2005. Constructions of intersubjectivity: Discourse, syntax, and cognition. Oxford: Oxford University Press.Search in Google Scholar

Walker, Marilyn, Pranav Anand, Robert Abbott and Ricky Grant. 2012a. Stance classification using dialogic properties of persuasion. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 592–596. Association for Computational Linguistics.Search in Google Scholar

Walker, Marilyn, Pranav Anand, Robert Abbott, Jean E. Fox Tree, Craig Martell and Joseph King. 2012b. That is your evidence?: Classifying stance in online political debate. Decision Support Systems 53 (4): 719–729.10.1016/j.dss.2012.05.032Open DOISearch in Google Scholar

Walker, Marilyn, Jean E. Fox Tree, Pranav Anand, Robert Abbott and Joseph King. 2012c. A corpus for research on deliberation and debate. In Proceedings of The Eighth International Conference on Language Resources and Evaluation (LREC), 812–817. Istanbul, Turkey.Search in Google Scholar

White, Peter R. 2003. Beyond modality and hedging: A dialogic view of the language of intersubjective stance. Text 23 (2): 259–284.10.1515/text.2003.011Search in Google Scholar

Wiebe, Janyce, Theresa Wilson and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation 39 (2): 165–210.10.1007/s10579-005-7880-9Open DOISearch in Google Scholar

Wiebe, Janyce, Theresa Wilson, Rebecca Bruce, Matthew Bell and Melanie Martin. 2004. Learning subjective language. Computational Linguistics 30 (3): 277–308.10.1162/0891201041850885Open DOISearch in Google Scholar

Zheng, Rong, Jiexun Li, Hisnchun Chen and Zan Huang. 2006. A framework for authorship identification of online messages: Writing-style features and classification techniques. Journal of the American Society for Information Science and Technology 57 (3): 378–393.10.1002/asi.20316Open DOISearch in Google Scholar