Examining validity in computerized dynamic assessment

Open access

Abstract

Computerized dynamic assessment (CDA) posits itself as a new type of assessment that includes mediation in the assessment process. Proponents of dynamic assessment (DA) in general and CDA in particular argue that the goals of DA are in congruence with the concept of validity that underscores the social consequences of test use and the integration of learning and assessment (Sternberg & Grigorenko, 2002; Poehner, 2008; Shabani, 2012;). However, empirical research on CDA falls short in supporting such an argument. Empirical studies on CDA are riddled with ill-defined constructs and insufficient supporting evidence in regard to the aspects of validity postulated by Messick (1989, 1990, 1996). Due to the scarcity of research on CDA, this paper explores the potentials and the viability of this intervention-based assessment in computer assisted language testing context in light of its conformity with Messick’s unitary view of validity. The paper begins with a discussion of the theoretical foundations and models of DA. It then proceeds to discuss the differences between DA and non-dynamic assessment (NDA) measures before critically appraising the empirical studies on CDA. The critical review of the findings in CDA literature aims at shedding light on some drawbacks in the design of CDA research and the compatibility of the concept of construct validity in CDA with Messick’s (1989) unitary concept of validity. The review of CDA concludes with some recommendations for rectifying gaps to establish CDA in a more prominent position in computerized language testing.

Aljaafreh, Ali, James P. Lantolf (1994). Negative feedback as regulation and second language learning in the zone of proximal development. The Modern Language Journal 78: 465–483.

Chapelle, Carol A. (2003). English Language Learning and Technology: Lectures on Applied Linguistics in the Age of Information and Communication Technology. Philadelphia, PA: John Benjamins.

Chapelle, Carol A., Dan Douglas (2006). Assessing Language Through Computer Technology. Cambridge: Cambridge University Press.

Birjandi, Parviz, Saman Ebadi (2012). Microgenesis in dynamic assessment of L2 learners’ sociocognitive development via web 2.0. Procedia - Social and Behavioral Sciences 32: 34–39.

Feuerstein, Reuven, Ya’acov Rand, John E. Rynders (1988). Don’t Accept Me As I Am: Helping “Retarded” People to Excel. New York: Plenum.

Frederiksen, John R., Alan Collins (1989). A systems approach to educational testing. Educational Researcher 18(9): 27-32.

Jacobs, Ellen L. (2001). The effects of adding dynamic assessment components to a computerized preschool language screening test. Communication Disorders Quarterly 22: 217–226.

Lantolf, James P., Matthew E. Poehner (2004). Dynamic assessment of L2 development: Bringing the past into the future. Journal of Applied Linguistics 1: 49–74.

Lantolf, James P., Steve L. Thorne (2006). Sociocultural Theory and the Genesis of Second Language Development. Oxford: Oxford University Press.

Lin, Zheng (2010). Interactive dynamic assessment with children learning EFL in kindergarten. Early Childhood Education Journal 37: 279–287.

Loevinger, Jane (1957). Objective tests as instruments of psychological theory. Psychological Reports 3: 635-694 (Monograph supplement 9).

McNamara, Tim (1997). “Interaction” in second language performance assessment. Whose performance? Applied Linguistics 18, 446-466.

McNamara, Tim (2001). Language assessment as social practice: Challenges for research. Language Testing 18(4): 333-349.

Messick, Samuel (1980). Test validity and ethics of assessment. American Psychologist, 35.11: 1012-1027.

Messick, Samuel (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher 18(2): 5-11.

Messick, Samuel (1990). Validity of test interpretation and use. Princeton, N.J: Educational Testing Service. ETS-RR-90-11

Messick, Samuel (1996). Validity and washback in language testing. Language Testing 13: 241-256.

Moss, Pamela A. (2003). Reconceptualizing validity for classroom assessment. Educational Measurement: Issues and practice 22.4: 13-25.

Poehner, Matthew E. (2007). Beyond the test: L2 dynamic assessment and the transcendence of mediated learning. The Modern Language Journal 91: 323–340.

Poehner, Matthew E. (2008). Dynamic Assessment: A Vygotskian Approach to Understanding and Promoting Second Language Development. Berlin: Springer Publishing.

Poehner, Matthew E. (2011). Validity and interaction in the ZPD: Interpreting learner development through L2 dynamic assessment. International Journal of Applied Linguistics 21(2): 244-263.

Poehner, Matthew E., James P. Lantolf (2013). Bringing the ZPD into the equation: Capturing L2 development during computerized dynamic assessment (CDA). Language Teaching Research, 17(3): 323–342.

Porte, Graeme Keith (2010). Appraising Research in Second Language Learning: A Practical Approach to Critical Analysis of Quantitative Research (2nd ed.). Amsterdam: John Benjamins.

Ratner, Carl (1997). Cultural Psychology: Theory and Methods. New York: Plenum.

Shabani, Karim (2012). Dynamic assessment of L2 learners’ reading comprehension processes: A Vygotskian perspective. Procedia - Social and Behavioral Sciences 32: 321-328.

Shrestha, Prithvi, Caroline Coffin (2012). Dynamic assessment, tutor mediation and academic writing development. Assessing Writing 17: 55–70.

Sternberg, Robert J., Elena L. Grigorenko (2002). Dynamic Testing: The Nature and Measurement of Learning Potential. Cambridge, UK: Cambridge University Press.

Teo, Adeline, Fu Jen (2012). Promoting EFL students’ inferential reading skills through computerized dynamic assessment. Language Learning and Technology, 16.3: 10-20.

Vygotsky, Lev S. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press.

ExELL

Explorations in English Language and Linguistics

Journal Information

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 99 99 24
PDF Downloads 24 24 8