Assessing the inter-coder reliability of the Body Type Dictionary (BTD)

Open access

Abstract

Computer-assisted content analysis has many advantages compared to a manual scoring system, provided that computerised dictionaries represent valid and reliable measures. This study aimed to assess the inter-coder reliability, alternate- form reliability and scoring consistency of the Body Type Dictionary (BTD) (Wilson 2006) based on Fisher and Cleveland’s (1956, 1958) manual body boundary scoring scheme. The results indicated an acceptable inter-coder agreement with barrier and penetration imagery in the sub-sample (N = 53) of manually coded Rorschach responses. Additionally manually coded scores showed an acceptable correlation with the computerised frequency counts, and thus indicating an alternate-form reliability. In the full data set (N = 526), barrier imagery in the Rorschach responses only correlated with the picture response test, showing low scoring consistency, which might disconfirm the notion of body boundary awareness representing a stable personality trait but instead it might be dependent on the level of cognitive dedifferentiation.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Ackerman Steven J. Mark J. Hilsenroth Amanda J. Clemence Robin Weatherill and I. Christopher Fowler. 2001. Convergent validity of Rorschach and TAT scales of object relations. Journal of Personality Assessment 77: 295-306.

  • Anzieu Didier. 1989. The skin ego. New Haven: Yale University Press.

  • Argamon Shlomo and Shlomo Levitan. 2005. Measuring the usefulness of function words for authorship attribution. Proceedings of the 2005 ACH/ ALLC conference June 2005. Victoria Canada.

  • Artstein Ron and Massimo Poesio. 2007. Inter-coder agreement for computational linguistics. Computational Linguistics 34: 555-596.

  • Bick Esther. 1968. Experience of the skin in early object relations. International Journal of Psycho-Analysis 49: 484-486.

  • Buchanan Tom. 2002. One assessment: Desirable or dangerous? Professional Psychology: Research and Practice 33: 148-154.

  • Buck Lucien A. and Michael Barden. 1971. Body image scores and varieties of consciousness. Journal of Personality Assessment 35: 309-314.

  • Burris Christopher T. and John K. Rempel. 2004. ‘It’s the end of the world as we know it’: Threat and the spatial-symbolic self. Journal of Personality and Social Psychology 86: 19-42.

  • Cariola Laura A. 2012. A case study of primary process language and body boundary imagery in discourses of religious-mystical and psychotic altered states of consciousness. Empirical Text and Cultural Research - ETC 5: 36-61.

  • Chang Cindy and James W. Pennebaker. 2007. The psychology of function words. In K. Fiedler (ed.). Social communication 343-359. New York: Psychology Press.

  • Fisher Seymour and Sidney E. Cleveland. 1956. Body-image boundaries and style of life. Journal of Abnormal and Social Psychology 52: 373-379.

  • Fisher Seymour and Sidney E. Cleveland. 1958. Body image and personality. New York: Dover Publications.

  • Fisher Seymour. 1986. Development and structure of the body image. Hillsdale: Lawrence Erlbaum.

  • Fleiss Joseph L. 1981. Statistical methods for rates and proportions. New York: John Wiley.

  • Fox John. 2005. The R commander: A basic statistics graphical user interface to R. Journal of Statistical Software 14: 1-42.

  • Friedman Milton. 1937. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association 32: 675-701.

  • Garmer Matthias Jim Lemon Ian Fellows and Suspendra Singh. 2012. Various coefficients of interrater reliability and agreement. Available at http://www.cran.r-project.org/web/packages/irr/irr.pdf. Last accessed on 19 January 2014.

  • Gigerenzer Gerd. 2004. Mindless statistics. The Journal of Socio-Economics 33: 587-606.

  • Guimon Jose. 1997. Corporality and psychoses. In J. Guimon (ed.). The body in psychotherapy 63-72. Basel: Karger.

  • Hogenraad Robert Claude Daubies Yves Bestgen and Pierre Mahau. 2003. Une théorie et une méthode générale d’analyse textuelle assistée par ordinateur: Le systéme PROTAN (PROTocol ANalyzer). 32-bits version of November 10 2003 by Pierre Mahau. Louvain-la-Neuve: Psychology Department Catholic University of Louvain.

  • Jackson Sherri. 2011. Research methods and statistics: A critical thinking approach. Belmont: Wadsworth.

  • Kolbe Richard H. and Melissa S. Burnett. 1991. Content-analysis research: An examination of applications with directives for improving research reliability and objectivity. Journal of Consumer Research 18: 243-250.

  • Krippendorff Klaus. 2004. Content analysis: An introduction to its methodology. Thousand Oaks: Sage. (First published 1980).

  • Lacy Stephen and Daniel Riffle. 1996. Sampling error and selecting intercoder reliability samples for nominal content categories: Sins of omission and commission in mass communication quantitative research. Journalism & Mass Communication Quarterly 73: 969-973.

  • Lombard Matthew Jennifer Snyder-Duch and Cheryl Campanella Bracken. 2002. Content analysis in mass communication: Assessment and reporting of inter-coder reliability. Human Communication Research 28: 587-604.

  • Lombard Matthew Jennifer Snyder-Duch and Cheryl Bracken. 2010. Practical resources for assessing and reporting intercoder reliability in content analysis research projects. Available at http://matthewlombard.com/reliability/. Last accessed on 29 January 2014.

  • Michalczyk Alan E. and Lloyd A. Lewis. 1980. Significance alone is not enough. Journal of Medical Education 55: 835-838.

  • Morgan Christina D. and Henry A. Murray. 1935. A method of investigating fantasies: The Thematic Apperception Test. Archives of Neurology and Psychiatry 34: 289-306.

  • Neuendorf Kimberly A. 2002. The content analysis guidebook. London: Sage Publications.

  • Neundorf Kimberly A. and Paul D. Skalsi. 2010. Extending the utility of content analysis via the scientific method. Manuscript in support of presentation to the Social Science Computing Workshop University of Hawaii Honolulu. Available at http://www.manoa.hawaii.edu/ccpv/workshops/KimberlyNeuendorf.pdf.

  • Ogden Thomas H. 1989. The primitive edge of experience. Northvale: Jason Aronson. O’Neill Regina M. 2005. Body image body boundary and the Barrier and Penetration Rorschach scoring system. In R. F. Bornstein and J. M. Masling (eds.). Scoring the Rorschach: Seven validated systems 159-189. London: Lawrence Erlbaum.

  • Orne Martin T. 1962. On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist 17: 776-783.

  • Passonneau Rebecca. 2006. Measuring agreement on set-valued items (MASI) for semantic and pragmatic annotation. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). Genoa Italy.

  • Potter W. James and Deborah Levine-Donnerstein. 1999. Rethinking validity and reliability in content analysis. Journal of Applied Communication Research 27: 258-284.

  • R Development Core Team. 2011. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Available at http://www.R-project.org. Last accessed on 19 January 2014.

  • Rorschach Hermann 1921. Psychodiagnostik. Leipzig: Ernst Bircher Verlag.

  • Rosenthal Robert and Ralph L. Rosnow. 1984. Essentials of behavioural research: Methods and data analysis. London: McGraw-Hill.

  • Rourke Liam Terry Anderson D. R. Garrison and Walter Archer. 2000. Methodology issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education 11: 8-22.

  • Saraceni Carlo Giuseppe Ruggeri and D. Filocamo. 1980. Studio sperimentale con il test di Rorschach sulle modificazioni dell’immagine corporea in ipnosi. Archivio di Psicologia Neurologia e Psichiatria 41: 50-64.

  • Schmeidler Gertrude and Lawrence LeShan. 1970. An aspect of body image related to ESP scores. Journal of the American Society for Psychical Research 64: 211-218.

  • Schmitt Norbert and Bruce Dunham. 1999. Exploring native and non-native intuitions of Word frequency. Second Language Research 15: 389-411.

  • Schnurr Paula P. Stanley D. Rosenberg Thomas E. Oxman and Gary J. Tucker. 1986. A methodological note on content analysis: Estimates of reliability. Journal of Personality Assessment 50: 601-609.

  • Singletary Michael W. 1994. Mass communication research: Contemporary methods and applications. New York: Longman. Spearman Charles. 1904. The proof and measurement of association between two things. American Journal of Psychology 15: 72-101.

  • Tinsley Howard. E. A. and David J. Weiss. 1975. Interrater reliability and agreement of subjective judgements. Journal of Counseling Psychology 22: 358-376.

  • Tustin Frances. 1981. Autistic states in children. London: Routledge.

  • Weber Robert P. 1990. Basic content analysis. London: Sage.

  • West Alan N. 1991. Primary process content in the King James Bible: The five stages of Christian mysticism. Computers and the Humanities 25: 227-238.

  • Wilson Andrew. 2006. The development and application of a content analysis dictionary for body boundary research. Literary and Linguistic Computing 21: 105-110.

  • Wilson Andrew. 2008. Psychosomatic cycles and the liturgical year: A case study and framework for research. Gottingen: Cuvillier Verlag.

  • Wilson Andrew. 2009. Barrier and penetration imagery in altered states of consciousness discourse: Replicating the five-stage model of Christian mysticism in the Bible. In W. Oleksy and P. Stalmaszczyk (eds.). Cognitive approaches to language and linguistic data: Studies in honor of Barbara Lewandowska-Tomaszczyk (Polish Studies in English Language and Literature 27) 357-372. Frankfurt am Main: Peter Lang.

  • Wilson Andrew. 2011. The regressive imagery dictionary: A test of its concurrent validity in English German Latin and Portuguese. Literary and Linguistic Computing 26: 125-135.

Search
Journal information
Metrics
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 239 126 2
PDF Downloads 106 66 1