Welcome to Bilingual Publishing Co.
+65 65881289
contact@bilpublishing.com

Improved Behavioural Engagement of Students through Low-stakes Online Tests and Immediate Dialogic Feedback

Glen Currie (The University of Melbourne)
Tracii Ryan (The University of Melbourne, Australia)

Article ID: 5295

DOI: https://doi.org/10.30564/jiep.v6i1.5295

Abstract


The engagement of students is a recognised challenge for teachers. Technology offers some practical student engagement tools, and this paper examines the use of low-stakes online tests and immediate dialogic feedback to improve behavioural engagement. The academic exploration of low-stakes tests and dialogic feedback has been extensive, and they are credible teaching tools. In this study, we explore the learning benefit of their combination. Postgraduate engineering students' self-reported and learning analytics data shows conclusive evidence of improved behavioural engagement. We measured a 500% increase in the Learning Management System (LMS) page views on the days when we ran the low-stakes tests (each worth 2% of the marks for the subject) and engaged in immediate dialogic feedback. To interpret these results, we draw on theories of behavioural engagement, low-stakes tests, and feedback. We conclude that the combination of low-stakes tests and immediate feedback improves student behavioural engagement.

Keywords


Behavioural engagement；Immediate dialogic feedback；Low-stakes tests；Mixed-methods; Online learning; Quantitative; Survey

Full Text:

PDF

References


[1] Aguilera-Hermida, A. P. (2020). College students’ use and acceptance of emergency online learning due to COVID-19. International Journal of Educational Research Open, 1, 100011. https://doi.org/10.1016/j.ijedro.2020.100011

[2] Askew, S., & Lodge, C. (2004). Gifts, ping-pong and loops–linking feedback and learning. In Feedback for learning (pp. 13-30). Routledge. https://doi.org/10.4324/9780203017678

[3] Bedenlier, S., Bond, M., Buntins, K., Zawacki-Richter, O., & Kerres, M. (2020). Facilitating student engagement through educational technology in higher education: A systematic review in the field of arts and humanities. Australasian Journal of Educational Technology, 36(4), 126-150. https://doi.org/10.14742/ajet.5477

[4] Boud, D., & Molloy, E. (2013). Feedback in higher and professional education: understanding it and doing it well. Routledge. https://doi.org/10.4324/9780203074336

[5] Broadbent, J., Panadero, E., & Boud, D. (2018). Implementing summative assessment with a formative flavour: a case study in a large class. Assessment & Evaluation in Higher Education, 43(2), 307-322. https://doi.org/10.1080/02602938.2017.1343455

[6] Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., & Rubin, G. J. (2020). The psychological impact of quarantine and how to reduce it: rapid review of the evidence. The Lancet. https://doi.org/10.1016/S0140-6736(20)30460-8

[7] Carless, D. (2012). Trust and its role in facilitating dialogic feedback. In Feedback in higher and professional education (pp. 100-113). Routledge. https://doi.org/10.4324/9780203074336

[8] Carless, D., & Chan, K. K. H. (2017). Managing dialogic use of exemplars. Assessment & Evaluation in Higher Education, 42(6), 930-941. https://doi.org/10.1080/02602938.2016.1211246

[9] Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. https://doi.org/03075071003642449

[10] Carless, D., & Winstone, N. (2020). Teacher feedback literacy and its interplay with student feedback literacy. Teaching in Higher Education, 1-14. https://doi.org/10.1080/13562517.2020.1782372

[11] Crawford, R., & Jenkins, L. (2017). Blended learning and team teaching: Adapting pedagogy in response to the changing digital tertiary environment. Australasian Journal of Educational Technology, 33(2). https://doi.org/10.14742/ajet.2924

[12] Dawson, P., Henderson, M., Ryan, T., Mahoney, P., Boud, D., Phillips, M., & Molloy, E. (2018). Technology and feedback design. Learning, Design, and Technology, Michael J Spector, Barbara B Lockee, and Marcus D. Childress (Eds.). Springer International Publishing, Cham, 1-45. https://doi.org/10.1007/978-3-319-17727-4_124-1

[13] Devlin, M., & McKay, J. (2016). Teaching students using technology: Facilitating success for students from low socioeconomic status backgrounds in Australian universities. Australasian Journal of Educational Technology, 32(1). https://doi.org/10.14742/ajet.2053

[14] Evans, T., Kensington-Miller, B., & Novak, J. (2021). Effectiveness, efficiency, engagement: Mapping the impact of pre-lecture quizzes on educational exchange. Australasian Journal of Educational Technology, 37(1), 163-177. https://doi.org/10.14742/ajet.6258

[15] Finley, S. (2019). Writing effective multiple choice questions. In Learning and Teaching in Higher Education. Edward Elgar Publishing. https://doi.org/10.4337/9781788975087.00046

[16] Forbes, H. a. M., Suzie. (2018). Professional development: Enhancing the MCQ item writing capability of academic staff. https://blogs.deakin.edu.au/deakin-learning-and-teaching-conference/wp-content/uploads/sites/319/2018/11/E1-S2-sp3-ForbesMacfarlane_ProfessionalDevelopmentMCQs_East1_1.40pm.pdf

[17] Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of educational research, 74(1), 59-109. https://doi.org/10.3102/00346543074001059

[18] Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112. https://doi.org/10.3102/003465430298487

[19] Higgins, R., Hartley, P., & Skelton, A. (2002). The conscientious consumer: Reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27(1), 53-64. https://doi.org/10.1080/03075070120099368

[20] Ingram, E. L., & Nelson, C. E. (2006). Using discussions of multiple choice questions to help students identify misconceptions & reconstruct their understanding. The American Biology Teacher, 68(5), 275-279. https://doi.org/10.2307/4451988

[21] Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38(5), 758-773. https://doi.org/10.1080/03075079.2011.598505

[22] Kuh, G. D. (2001). The National Survey of Student Engagement: Conceptual framework and overview of psychometric properties. https://nsse.indiana.edu/nsse/about-nsse/conceptual-framework/index.html

[23] Kulik, J. A., & Kulik, C.-L. C. (1988). Timing of feedback and verbal learning. Review of educational research, 58(1), 79-97. https://doi.org/10.3102/00346543058001079

[24] Laurillard, D. (2013). Rethinking university teaching: A conversational framework for the effective use of learning technologies. Routledge. https://doi.org/10.4324/9780203160329

[25] Lewin, K. (1946). Action research and minority problems. Journal of social issues, 2(4), 34-46. https://doi.org/10.1111/j.1540-4560.1946.tb02295.x

[26] Malau-Aduli, B. S., Assenheimer, D., Choi-Lundberg, D., & Zimitat, C. (2014). Using computer-based technology to improve feedback to staff and students on MCQ assessments. Innovations in Education and Teaching International, 51(5), 510-522. https://doi.org/10.1080/14703297.2013.796711

[27] Maringe, F., & Sing, N. (2014). Teaching large classes in an increasingly internationalising higher education environment: Pedagogical, quality and equity issues. Higher Education, 67(6), 761-782. https://doi.org/10.1007/s10734-013-9710-0

[28] McKenney, S., & Reeves, T. C. (2018). Conducting educational design research. Routledge.

[29] Miller, T. (2009). Formative computer‐based assessment in higher education: The effectiveness of feedback in supporting student learning. Assessment & Evaluation in Higher Education, 34(2), 181-192. https://doi.org/10.1080/02602930801956075

[30] Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and higher Education, 31(1), 53-64. https://doi.org/10.1080/03098770601167922

[31] Perusall. (2021). Perusall. Retrieved 2 June 2021 from www.perusall.com.

[32] Ramsden, P. (2003). Learning to teach in higher education. Routledge. https://doi.org/10.4324/9780203507711

[33] Reeves, T. C. (2015). Educational design research: Signs of progress. Australasian Journal of Educational Technology, 31(5). https://doi.org/10.14742/ajet.2902

[34] Roediger III, H. L. (2013). Applying cognitive psychology to education: Translational educational science. Psychological Science in the Public Interest, 14(1), 1-3. https://doi.org/10.1177/1529100612454415

[35] Ryan, T., French, S., & Kennedy, G. (2019). Beyond the Iron Triangle: improving the quality of teaching and learning at scale. Studies in Higher Education, 1-12. https://doi.org/10.1080/03075079.2019.1679763

[36] Sancho-Vinuesa, T., Escudero-Viladoms, N., & Masià, R. (2013). Continuous activity with immediate feedback: A good strategy to guarantee student engagement with the course. Open Learning: The Journal of Open, Distance and e-Learning, 28(1), 51-66. https://doi.org/10.1080/02680513.2013.776479

[37] Shuhidan, S., Hamilton, M., & D'Souza, D. (2010). Instructor perspectives of multiple-choice questions in summative assessment for novice programmers. Computer Science Education, 20(3), 229-259. https://doi.org/10.1080/08993408.2010.509097

[38] Shute, V. J., & Kim, Y. J. (2014). Formative and stealth assessment. In Handbook of research on educational communications and technology (pp. 311-321). Springer. https://doi.org/10.1007/978-1-4614-3185-5_25

[39] Sinfield, D., & Cochrane, T. (2020). A framework for rethinking the pedagogy of studio-based design classrooms. Pacific Journal of Technology Enhanced Learning, 2(2), 31-44. https://doi.org/10.24135/pjtel.v2i2.77

[40] Steen-Utheim, A., & Wittek, A. L. (2017). Dialogic feedback and potentialities for student learning. Learning, Culture and Social Interaction, 15, 18-30. https://doi.org/10.1016/j.lcsi.2017.06.002

[41] Sugden, N., Brunton, R., MacDonald, J., Yeo, M., & Hicks, B. (2021). Evaluating student engagement and deep learning in interactive online psychology learning activities. Australasian Journal of Educational Technology, 37(2), 45-65. https://doi.org/10.14742/ajet.6632

[42] Thomas, L. (2012). Building student engagement and belonging in Higher Education at a time of change. https://www.heacademy.ac.uk/system/files/what_works_final_report_0.pdf

[43] van Alten, D. C., Phielix, C., Janssen, J., & Kester, L. (2019). Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis. Educational Research Review, 28, 100281. https://doi.org/10.1016/j.edurev.2019.05.003

[44] Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of educational research, 85(4), 475-511. https://doi.org/10.3102/0034654314564881

[45] Vygotsky, L. S., & Cole, M. (1978). Mind in society: Development of higher psychological processes. Harvard university press.

[46] Wenger, E. (2009). Communities of practice. Communities, 22(5), 57-80. https://doi.org/10.1017/CBO9780511803932

[47] Whitmer, B., & Daley, D. (2020). Canvas LMS. In Infrastructure.

[48] Williams, B., Brown, T., & Benson, R. (2012). Feedback in the digital environment. In Feedback in higher and professional education (pp. 135-149). Routledge. https://doi.org/10.4324/9780203074336-13

[49] Willis, J., Gibson, A., Kelly, N., Spina, N., Azordegan, J., & Crosswell, L. (2021). Towards faster feedback in higher education through digitally mediated dialogic loops. Australasian Journal of Educational Technology, 22-37. https://doi.org/10.14742/ajet.5977

[50] Wise, S. L., & DeMars, C. E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational assessment, 10(1), 1-17. https://doi.org/10.1207/s15326977ea1001_1

[51] Yang, M., & Carless, D. (2013). The feedback triangle and the enhancement of dialogic feedback processes. Teaching in Higher Education, 18(3), 285-297. https://doi.org/10.1080/13562517.2012.719154


Refbacks

  • There are currently no refbacks.