Investigating disengagement is a continuing concern within computer-based learning environments. Drawing upon several strands of research into preservice teacher learning with network-based tutors, this paper outlines an object orientation to conceptualize a type of disengaged behaviour referred to as carelessness. We further differentiate this construct in terms of carelessness towards one’s own learning as opposed to other’s learning. In support of our claims, we review research into carelessness in the context of nBrowser, an intelligent web browser designed to support preservice teachers learn about the pedagogical affordances of novel technologies while designing lesson plans. The key aspects of this research can be listed as follows: (1) a knowledge engineering approach to implement a set of production rules within the learning environment to detect instances of carelessness and intervene; (2) a data-driven approach to infer learner behaviours in their absence due to carelessness; and (3) a model-driven approach to improve the functioning of the learning environment despite instances of carelessness. We discuss the limitations of these different approaches and draw implications for future research into preservice teacher disengagement with computer-based learning environments.
If the inline PDF is not rendering correctly, you can download the PDF file here.
Aleven, V., McLaren, B. M., Roll, I. & Koedinger, K. R. (2004). Toward tutoring help seeking: Applying cognitive modeling to meta-cognitive skills. In Proceedings of the 7th International Conference on Intelligent Tutoring Systems (pp. 227-239).
Aleven, V., McLaren, B., Roll, I. & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. International Journal of Artificial Intelligence in Education, 16, 101-130.
Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Meheranian, H., Fisher, D., Barto, A., Mahadevan, S. & Woolf. B. P. (2007). Repairing disengagement with non-invasive interventions. In Proceedings of the 13th International Conference on Artificial Intelligence in Education (pp. 195-202). Springer, Berlin, Heidelberg.
Azevedo, R., Moos, D. C., Johnson, A. M., & Chauncey, A. D. (2010). Measuring cognitive and metacognitive regulatory processes during hypermedia learning: Issues and challenges. Educational Psychologist, 45(4), 210-223. http://dx.doi.org/10.1080/00461520.2010.515934
Baker, R. S.J.d., & Gowda, S. M. (2010). An analysis of the differences in the frequency of students’ disengagement in urban, rural, and suburban high schools. In Proceedings of the 3rd International Conference on Educational Data Mining (pp. 11-20).
Baker, R. S.J.d. (2007a). Modeling and understanding students’ off-task behavior in intelligent tutoring systems. In Proceedings of ACM CHI 2007: Computer-Human Interaction (pp. 1059-1068).
Baker, R. S.J.d. (2007b). Is gaming the system state-or-trait? Educational data mining through the multi-contextual application of a validated behavioral model. In Complete On-Line Proceedings of the Workshop on Data Mining for User Modeling at the 11th International Conference on User Modeling 2007 (pp. 76-80).
Baker, R. S.J.d., Corbett, A. T., Koedinger, K. R., Evenson, S. E., Roll, I., Wagner, A. Z., Naim, M., Raspat, J., Baker, D. J. & Beck, J. (2006). Adapting to when students game an intelligent tutoring system. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 392-401).
Baker, R. S.J.d, Corbett, A. T., Koedinger, K. R. & Wagner, A. Z. (2004). Off-task behavior in the Cognitive Tutor classroom: When students “game the system”. In Proceedings of ACM CHI 2004: Computer-Human Interaction (pp. 383-390).
Baker, R. S.J.d., Rossi, L.M. (2013) Assessing the Disengaged Behaviors of Learners. In Sottilare, R., Graesser, A., Hu, X., & Holden, H. (Eds.) Design Recommendations for Adaptive Intelligent Tutoring Systems – Volume 1 – Learner Modeling. U.S. Army Research Lab, Orlando, FL, pp. 155-166, 2013.
Beal, C. R., Qu, L. & Lee, H. (2006). Classifying learner engagement through integration of multiple data sources. In Proceedings of the 21st National Conference on Artificial Intelligence (pp. 2-8). AAAI (American Association for Artificial Intelligence) Press.
Beck, J. (2005). Engagement tracing: using response times to model student disengagement. In Proceedings of the 12th International Conference on Artificial Intelligence in Education (pp. 88-95). Springer, Berlin, Heidelberg.
Buckley, B., Gobert, J. & Horwitz, P. (2006). Using log files to track students’ model-based inquiry. In Proceedings of the Seventh International Conference of the Learning Sciences (pp. 57-63). International Society of the Learning Sciences.
Cocea, M., Hershkovitz, A. & Baker, R. S.J.d. (2009). The impact of off-task and gaming behaviors on learning: Immediate or aggregate? In Proceedings of the 14th International Conference on Artificial Intelligence in Education (pp. 507-514). Springer, Berlin, Heidelberg.
D’Mello, S., & Graesser, A.C. (2011). The Half-Life of Cognitive-Affective States during Complex Learning. Cognition and Emotion, 25(7), 1299-1308.
Fredricks, J. A., Blumenfeld, P. C. & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109.
Gong, Y., Beck, J. E. & Heffernan, N. T. (2010). Comparing knowledge tracing and performance factor analysis by using multiple model fitting procedures. In Proceedings of the 10th International Conference on Intelligent Tutoring Systems (pp. 35-44).
Greer, J. & Mark, M. (2015). Evaluation Methods for Intelligent Tutoring Systems Revisited. International Journal of Artificial Intelligence in Education, 389-390.
Hannafin, M. J. (1995). Open-ended learning environments: Foundations, assumptions, and implications for automated design. In R. Tennyson, A.E. Barron (Ed.), Automating Instructional Design: Computer-Based Development and Delivery Tools (pp. 101-129). New York: Springer-Verlag.
Hershkovitz, A., Baker, R. S.J.d., Gobert, J., Wixon, M., Sao Pedro, M. (2013). Discovery with Models: A Case Study on Carelessness in Computer-based Science Inquiry. American Behavioral Scientist, 57(10), 1480-1499.
Hill, J., & Land, S. (1998). Open-ended learning environments: A theoretical framework and model for design. In M. Simonsen (Ed.), Proceedings of Selected Research and Development Presentations at the National Convention of the Association for Educational Communications and Technology. St Louis, MO: AECT. ERIC Document Reproduction Service No. ED423839.
Johns, J., & Woolf, B. (2006). A dynamic mixture model to detect student motivation and proficiency. In Proceedings of the 21st National Conference on Artificial Intelligence (pp. 163-168). AAAI (American Association for Artificial Intelligence) Press.
Lajoie, S. P., & Azevedo, R. (2006). Teaching and learning in technology-rich environments. In P. Alexander & P. Winne (Eds.), Handbook of educational psychology (2nd ed.) (pp. 803-821). Mahwah, NJ: Erlbaum.
Land, S. M. (2000). Cognitive requirements for learning with open-ended learning environments. Educational Technology Research and Development, 48(3), 61-78.
McNamara, D. S., Crossley, S. A., & Roscoe, R. (2013). Natural language processing in an intelligent writing strategy tutoring system. Behavior Research Methods, 45(2), 499-515. doi: 10.3758/s13428-012-0258-1.
Muldner, K., Burleson, W., Van de Sande, B. & VanLehn, K. (2011). An analysis of students’ gaming behaviors in an intelligent tutoring system: Predictors and impacts. User Modeling and User-Adapted Interaction, 21(1-2), 99-135.
Pekrun, R., & Linnenbrink-Garcia, L. (2014). Introduction to emotions in education. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), Educational psychology handbook series. International handbook of emotions in education (pp. 1-10). New York, NY, US: Routledge/Taylor & Francis Group.
Poitras, E. G., Doleck, T., & Lajoie, S. P. (2018). Modeling student teachers’ information seeking behaviors: Implications for adaptive scaffolding of lesson planning. Paper presented at the 2018 American Educational Research Association Annual Meeting. New York, NY.
Poitras, E. G., & Fazeli, N. (2017). Simulating preservice teachers’ information-seeking behaviors while learning with an intelligent web browser. Paper presented at the 2017 Society for Information Technology and Teacher Educational annual conference. Austin, TX.
Poitras, E., & Fazeli, N. (2016). Mining the Edublogosphere to Enhance Teacher Professional Development. In Shalin Hai-Jew (Ed.), Social Media Data Extraction and Content Analysis. IGI Global.
Poitras, E., Doleck, T., Huang, L., Li, S., & Lajoie, S. (2017). Advancing Teacher Technology Education Using Open-Ended Learning Environments as Research and Training Platforms. Australian Journal of Educational Technology, 33(3), 32-45.
Poitras, E., Doleck, T., Huang, L., Li, S., & Lajoie, S. (2018). nBrowser: An Intelligent Web Browser for Studying Self-Regulated Learning in Teachers’ Use of Technology. In Robert Zheng (Ed.), Strategies for Deep Learning with Digital Technology: Theories and Practices in Education. Nova Science Publishers.
Poitras, E., Fazeli, N., & Mayne, Z. (2018). Modeling Student Teachers’ Information-Seeking Behaviors while Learning with Network-Based Tutors. Journal of Educational Technology & Society.
Poitras, E., Mayne, Z., Huang, L., Doleck, T., Udy, L., & Lajoie, S. (2018). Simulated student behaviors with intelligent tutoring systems: Applications for authoring and evaluating network-based tutors. In Scotty Craig (Ed.), Tutoring and Intelligent Tutoring Systems. Nova Publishers.
Rowe, J., McQuiggan, S., Robison, J. & Lester, J. (2009). Off-task behavior in narrative-centered learning environments. In Proceedings of the 14th International Conference on Artificial Intelligence in Education (pp. 99-106). Springer, Berlin, Heidelberg.
Sabourin, J., Rowe, J., Mott, B. & Lester, J. (2011). When off-task is on-task: The affective role of off-task behavior in narrative-centered learning environments. In Proceedings of the 15th International Conference on Artificial Intelligence in Education (pp. 534-536). Springer, Berlin, Heidelberg.
Shute, V. J., & Zapata-Rivera, D. (2012). Adaptive educational systems. In P. Durlach (Ed.), Adaptive technologies for training and education (pp. 7-27). New York, NY: Cambridge University Press.
Sottilare, R., Graesser, A., Hu, X., & Holden, H. (Eds.). (2013). Design recommendations for Intelligent Tutoring Systems. Orlando, FL: U.S. Army Research Laboratory.
Vanlehn, K., Ohlsson, S., & Nason, R. (1994). Applications of simulated students: An exploration. Journal of Artificial Intelligence in Education, 5(2), 135-175.
Walonoski, J. A. & Heffernan, N. T. (2006). Prevention of off-task gaming behavior in intelligent tutoring systems. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 722-724). Springer-Verlag: Berlin.
Wixon, M., d Baker, R. S., Gobert, J. D., Ocumpaugh, J., & Bachmann, M. (2012, July). WTF? detecting students who are conducting inquiry without thinking fastidiously. In Proceedings of the 20th International Conference on User Modeling, Adaptation, and Personalization (pp. 286-296). Springer, Berlin, Heidelberg.