This paper describes a design-based implementation research (DBIR) approach to the development and trialling of a new generation massive open online course (ngMOOC) situated in an instructional setting of undergraduate mathematics at a regional Australian university. This process is underscored by two important innovations: (a) a basis in a well-established human cognitive architecture in terms of cognitive load theory; and (b) point-of-contact feedback based in a well-tested online system dedicated to enhancing the learning process. Analysis of preliminary trials suggests that the DBIR approach to the ngMOOC construction and development supports theoretical standpoints that argue for an understanding of how design for optimal learning can utilise conditions, such as differing online or blended educational contexts, in order to be effective and scalable. The ngMOOC development described in this paper marks the adoption of a cognitive architecture in conjunction with feedback systems, offering the groundwork for use of adaptive systems that cater for learner expertise. This approach seems especially useful in constructing and developing online learning that is self-paced and curriculum-based.
If the inline PDF is not rendering correctly, you can download the PDF file here.
1. Australian Academy of Science (2016). The mathematical sciences in Australia: A vision for 2025. Canberra, Australia: Australian Academy of Science.
2. Bali, M. (2014). MOOC pedagogy: gleaning good practice from existing MOOCs. Journal of Online Learning and Teaching, 10, 44-56.
3. Biggs, J. (1987). The study process questionnaire (SPQ): Manual. British Journal of Educational Psychology, 68, 395-407.
4. Biggs, J. (1999). What the student does: Teaching for enhanced learning. Higher Education Research & Development, 18, 57-75.
5. Biggs, J., Kember, D., & Leung, D. Y. (2001). The revised two-factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133-149
6. Boyd, W., Foster, A., Smith, J., & Boyd, W. E. (2014). Feeling good about teaching mathematics: Addressing anxiety amongst pre-service teachers. Creative Education, 5, 207-217.
7. Bozkurt, A., Akgün-Özbek, E., & Zawacki-Richter, O. (2017). Trends and patterns in Massive Open Online Courses: Review and content analysis of research on MOOCs (2008-2015). The International Review of Research in Open and Distributed Learning, 18(5). doi.org/10.19173/irrodl.v18i5.3080.
8. Bralić, A., & Divjak, B. (2018). Integrating MOOCs in traditionally taught courses: achieving learning outcomes with blended learning. International Journal of Educational Technology in Higher Education, 15(1). doi: 10.1186/s41239-017-0085-7.
9. Bressoud, D. M. (2014). Attracting and retaining students to complete two-and four-year undergraduate degrees in STEM: The role of undergraduate mathematics education. Commissioned paper prepared for the Committee on Barriers and Opportunities in Completing 2-Year and 4-Year STEM Degrees. Washington, DC: National Academy of Sciences.
10. Brookhart, S. M. (2008). How to give effective feedback to your students. Alexandria, VA: Association of Supervision and Curriculum Development.
11. Bruer, J. T. (2016). Where Is Educational Neuroscience? Educational Neuroscience, 1, 1-13.
12. Burdman, P. (2015). Degrees of freedom: Diversifying math requirements for college readiness and graduation. Oakland, CA: Learning Works and Policy Analysis for California Education.
14. Burnheim, C., & Harvey, A. (2016). Far from the studying crowd? Regional and remote students in higher education. In A. Harvey, C. Burnheim & M. Brett (Eds.), Student equity in Australian higher education (pp. 143-162). Singapore: Springer.
15. Champaign, J., Colvin, K. F., Liu, A., Fredericks, C., Seaton, D., & Pritchard, D. E. (2014). Correlating skill and improvement in 2 MOOCs with a student’s time on tasks. Proceedings of the first ACM conference on Learning @ scale conference, ACM, New York, USA, 11-20.
16. Chen, O., Woolcott, G., & Sweller, J. (2017). Using cognitive load theory to structure MOOCs and other computer-based learning. Journal of Computer Assisted Learning, 33(4), 293-305. doi:10.1111/jcal.12188
17. Chubb, I., Findlay, C., Du, L., Burmester, B., & Kusa, L. (2012). Mathematics, engineering and science in the national interest. Canberra, Australia: Office of the Chief Scientist.
18. Clarà, M., & Barberà, E. (2014). Three problems with the connectivist conception of learning. Journal of Computer Assisted Learning, 30, 197-206.
19. Croft, A., Harrison, M., & Robinson, C. (2009). Recruitment and retention of students–an integrated and holistic vision of mathematics support. International Journal of Mathematical Education in Science and Technology, 40, 109-125.
20. Daza, V., Makriyannis, N., & Rovira Riera, C. (2013). MOOC attack: closing the gap between pre-university and university mathematics. Open Learning: The Journal of Open, Distance and e-Learning, 28(3), 227-238.
21. Deloitte Report (2012). Measuring the economic benefits of mathematical science research in the UK. London, UK: Deloitte MCS Ltd.
22. DiSalvo, C. (2017). Viewing participatory design from the learning sciences and the field of design. In B. DiSalvo, J. Yip, E. Bonsignore & C. DiSalvo (Eds.), Participatory design for learning: Perspectives from practice and research (pp. 28-42). New York, NY: Routledge.
23. Diseth, A. (2003). Personality and approaches to learning as predictors of academic achievement. European Journal of Personality, 17, 143-155.
24. Dollinger, M., Lodge, J., & Coates, H. (2018). Co-creation in higher education: Towards a conceptual model. Journal of Marketing for Higher Education, 28(2), 1-22. doi: 10.1080/08841241.2018.1466756.
25. Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109-132.
26. El-Hmoudova, D. (2014). MOOCs motivation and communication in the cyber learning environment. Procedia-Social and Behavioral Sciences, 131, 29-34.
27. Entwistle, N., Tait, H., & McCune, V. (2000). Patterns of response to an approaches to studying inventory across contrasting groups and contexts. European Journal of Psychology of Education, 15, 33-48.
29. Fishman, B. J., Penuel, W. R., Allen, A. R., Cheng, B. H., & Sabelli, N. O. R. A. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. National Society for the Study of Education, 112, 136-156.
30. Freitas, A., & Paredes, J. (2018). Understanding the faculty perspectives influencing their innovative practices in MOOCs/SPOCs: a case study. International Journal of Educational Technology in Higher Education, 15(1). doi: 10.1186/s41239-017-0086-6.
31. Galligan, L. (2013). A systematic approach to embedding academic numeracy at university. Higher Education Research & Development, 32, 734-747.
32. Gašević, D., Kovanović, V., Joksimović, S., & Siemens, G. (2014). Where is research on massive open online courses headed? A data analysis of the MOOC Research Initiative. The International Review of Research in Open and Distributed Learning, 15(5).
33. Geary, D. C. (2012). Evolutionary educational psychology. In K. Harris, S. Graham, & T. Urdan (Eds.), APA Educational Psychology Handbook (Vol. 1, pp. 597-621). Washington, D.C.: American Psychological Association.
34. Groen, L., Coupland, M., Langtry, T., Memar, J., Moore, B., & Stanley, J. (2015). The mathematics problem and mastery learning for first-year, undergraduate STEM students. International Journal of Learning, Teaching and Educational Research, 11, 141-160.
35. Hanushek, E., & Woessmann, L. (2010). The High Cost of Low Educational Performance: The Long-Run Economic Impact of Improving PISA Outcomes. Paris, France: Author.
36. Hew, K. F. (2015). Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCs. British Journal of Educational Technology, 47, 320-341.
37. Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42, 371-406.
38. Holdren, J., & Lander, E. (2012). Report to the President – Engage to excel: Producing one million additional college graduates with degrees in Science, Technology, Engineering, and Mathematics. Washington, DC: President’s Council of Advisors on Science and Technology:
39. Kalyuga, S., Chandler, P., & Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology, 92, 126-136.
40. Kay, R., & Kletskin, I. (2012). Evaluating the use of problem-based video podcasts to teach mathematics in higher education. Computers & Education, 59, 619-627.
41. Keast, R., & Mandell, M. (2014). The collaborative push: Moving beyond rhetoric and gaining evidence. Journal of Management and Governance, 18, 9-28.
42. Kennedy, J. P., Lyons, T., & Quinn, F. (2014). The continuing decline of science and mathematics enrolments in Australian high schools. Teaching Science, 60, 34-46.
43. King, D., & Cattlin, J. (2015). The impact of assumed knowledge entry standards on undergraduate mathematics teaching in Australia. International Journal of Mathematical Education in Science and Technology, 46(7), 1032-1045.
44. Knox, J. (2016). Posthumanism and the massive open online course. New York, NY: Routledge.
46. Lake, W., Boyd, W., Boyd, W., & Hellmundt, S. (2017). Just another student survey? Point of contact survey feedback enhances the student experience and lets researchers gather data. Australian Journal of Adult Learning, 57, 82-104.
47. Lake, W., Wallin, M., Boyd, W. E., Woolcott, G., Boyd, W., Foster, A., & Markopoulos, C. (2018). Optimising the efficacy of hybrid academic teams: Lessons from a systematic review process. Australian Universities’ Review, 60(1), 16-24.
48. Lake, W., Wallin, M., Woolcott, G., Boyd, W. E., Foster, A., Markopoulos, C., & Boyd, W. (2017). Applying an alternative mathematics pedagogy for students with weak mathematics: Meta-analysis of alternative pedagogies. International Journal of Mathematical Education in Science and Technology, 48, 215-228.
49. Lakhani, J., Benzies, K., & Hayden, K. A. (2012). Attributes of interdisciplinary research teams: A comprehensive review of the literature. Clinical & Investigative Medicine, 35(5), E260-E265.
50. Lawson, D., Croft, T., & Waller, D. (2012). Mathematics support past, present and future. Proceedings of the International Conference on Innovation, Practice and Research in Engineering Education, 18-20. Loughborough University, UK: Centre for Engineering and Design Education.
51. Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distributed Learning, 14, 202-227.
52. Lyons, T., Cooksey, R., Panizzon, D., Parnell, A., & Pegg, J. (2006). Science, ICT and mathematics education in rural and regional Australia: The SiMERR national survey. A research report prepared for the Department of Education, Science and Training. Armidale, Australia: National Centre of Science, ICT and Mathematics Education for Rural and Regional Australia, and the University of New England.
58. Means, B., & Anderson, K. (2013). Expanding evidence approaches for learning in a digital world. Washington, DC: US Department of Education Office of Educational Technology.
59. van Merriënboer, J. J., & De Croock, M. B. (1992). Strategies for computer-based programming instruction: Program completion vs. program generation. Journal of Educational Computing Research, 8, 365-394.
60. Mesoudi, A. (2016). Cultural evolution: Integrating psychology, evolution and culture. Current Opinion in Psychology, 7, 17-22.
61. Moe, R. (2015). MOOCs as a Canary: A Critical Look at the Rise of EdTech. Proceedings of the E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (Vol. 2015, No. 1, pp. 1037-1042). Hawaii: Association for the Advancement of Computing in Education (AACE), Chesapeake, VA
62. Office of the Chief Scientist (2012). Mathematics, engineering and science in the national interest. Canberra, Australia: Office of the Chief Scientist.
63. Office of the Chief Scientist (2016). Australia’s STEM Workforce: Science, Technology, Engineering and Mathematics. Canberra, Australia: Australian Government.
64. Okimoto, H., & Heck, R. (2015). Examining the impact of redesigned developmental math courses in community colleges. Community College Journal of Research and Practice, 39, 633-646.
65. Paas, F. G., & van Merriënboer, J. J. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122-133.
66. Parikh, A., McReelis, K., & Hodges, B. (2001). Student feedback in problem based learning: A survey of 103 final year students across five Ontario medical schools. Medical Education, 35, 632-636.
67. Penney, C. G. (1989). Modality effects and the structure of short-term verbal memory. Memory & Cognition, 17, 398-422.
68. Penuel, W. R., Allen, A. R., Coburn, C. E., & Farrell, C. (2015). Conceptualizing research– practice partnerships as joint work at boundaries. Journal of Education for Students Placed at Risk (JESPAR), 20, 182-197.
69. Penuel, W. R., Coburn, C. E., & Gallagher, D. J. (2013). Negotiating problems of practice in research-practice design partnerships. Yearbook of the National Society for the Study of Education, 112, 237-255.
70. Penuel, W. R., Fishman, B. J., Cheng, B. H., & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational researcher, 40, 331-337.
71. Penuel, W. R., Fishman, B. J., Cheng, B. H., & Sabelli, N. (2016). Organizing research and development at the intersection of learning, implementation, and design. Annual Review of Policy Design, 4, 1-10.
72. Penuel, W. R., Roschelle, J., & Shechtman, N. (2007). Designing formative assessment software with teachers: An analysis of the co-design process. Research and Practice in Technology Enhanced Learning, 2, 51-74.
73. Peters, M. L. (2013). Examining the relationships among classroom climate, self-efficacy, and achievement in undergraduate mathematics: A multi-level analysis. International Journal of Science and Mathematics Education, 11, 459-480.
74. Petronzi, D., & Hadi, M. (2016). Exploring the factors associated with MOOC engagement, retention and the wider benefits for learners. European Journal of Open, Distance and E-learning, 19(2), 112-129.
75. Renkl, A., Atkinson, R. K., & Große, C. S. (2004). How fading worked solution steps works– a cognitive load perspective. Instructional Science, 32, 59-82.
76. Richardson, J. T. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment & Evaluation in Higher Education, 30, 387-415.
78. Scott, A., Woolcott, G., Keast, R., & Chamberlain, D. (2018). Sustainability of collaborative networks in higher education research projects: Why complexity? Why now? Public Management Review, 20(7), 1068-1087.
80. Siemens, G. (2013). Massive open online courses: Innovation in education. Open Educational Resources: Innovation, Research and Practice, 5, 5-15.
81. Steffens, K., Bannan, B., Dalgarno, B., Bartolomé, A. R., Esteve-González, V., & Cela-Ranilla, J. M. (2015). Recent developments in technology-enhanced learning: A critical assessment. RUSC: Universities and Knowledge Society Journal, 12(2). 73-86.
82. Stone, M. L., Kent, K. M., Roscoe, R. D., Corley, K. M., Allen, L. K., & McNamara, D. S. (2017). The design implementation framework. In R. D. Roscoe, S. D. Craig & I. Douglas (Eds.), End-User considerations in educational technology design (pp. 76-98). Hershey, PA: IGI Global.
83. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York, NY: Springer.
84. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 59-89.
85. Verstegen, D. M., Spruijt, A., Dolmans, D., & van Merriënboer, J. J. (2016). Problem-based learning in a MOOC – Exploring an innovative instructional design at a large scale. Proceedings of the 8th International Conference on Computer Supported Education – Volume 2, CSEDU, 369-377. doi: 10.5220/0005757003690377.
86. Waldrop, M. M. (2013). Campus 2.0. Nature, 495(7440), 160-163.
87. Watson, S. (2003). Closing the feedback loop: Ensuring effective action from student feedback. Tertiary Education and Management, 9, 145.
88. Woolcott, G., & Chamberlain, D. (2018). Measuring a university-community collaboration using social network analysis. International Journal of Learning and Change, 11(1), 18.
89. Woolcott, G., Chamberlain, D., Whannell, R., & Galligan, L. (2018a). Examining undergraduate student retention in mathematics using network analysis and relative risk. International Journal of Mathematical Education in Science and Technology TMES, 50(3). doi: 10.1080/0020739X.2018.1520932.
90. Woolcott, G., Keast, R., Tsasis, P., Charles, M., Farr-Wharton, B., Kivits, R., & Chamberlain, D. (2018b). A network connectivity framework for person-centred service models. Panel paper presented at the 22nd Annual International Research Society for Public Management Conference, IRSPM2018, University of Edinburgh Business School, Edinburgh, UK, 11-13 April 2018.
91. Woolcott, G., Mason, R., Markopoulos, C., Boyd, W., Chen, O., Seton, C., Lake, W., Whannell, R., Foster, A., Galligan, L., Marshman, M., Schmalz, J., & Sultanova, N. (2017a). Bite size maths—Building mathematics low SES student capability in regional/remote Australia. Final Report 2017 for the Higher Education Participation and Partnerships Programme (HEPPP) 2015 National Priorities Pool, Australian Government Department of Education and Training. Canberra, Australia: Australian Government.
92. Woolcott, G., Scott, A., Norton, M., Whannell, R., Galligan, L., Marshman, M., Pfeiffer, L., & Wines, C. (2017b). It’s part of my life: Engaging university and community to enhance science and mathematics education. Final report for Enhancing the Training of Mathematics and Science Teachers. Canberra, Australia: Department of Education and Training.
93. Woolcott, G., Scott, A., Norton, M., Whannell, R., Galligan, L., Marshman, M., Pfeiffer, L., & Wines, C. (2017c). The Enhancement-Lesson-Reflection process: A resource manual for science and mathematics learning and teaching. Companion Report to the Final report: It’s part of my life: Engaging university and community to enhance science and mathematics education. Canberra, Australia: Department of Education and Training.
94. Yin, R. K. (2013). Case study research: Design and methods. Thousand Oaks, CA: Sage.
95. Zheng, S., Rosson, M. B., Shih, P. C., & Carroll, J. M. (2015). Understanding student motivation, behaviors and perceptions in MOOCs. Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 1882-1895. ACM.