Tracking students’ progression in developing understanding of energy using AI technologies

Abstract

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] Students struggle to acquire the needed energy understanding to meaningfully participate in the energy discourse about socially relevant topics, such as energy transformation or climate change. Identifying students on differing learning trajectories, as well as differences in knowledge used, is essential to help students achieve the needed energy understanding. Collecting and analyzing the longitudinal and fine-grained data necessary for this represents a substantial challenge. However, the use of a digital workbook, which captures all interaction data, has enabled us to collect such data from 𝑁=548 students (data from 172 students were analyzed after applying exclusion criteria). Using machine learning and natural language processing, we analyzed the data to identify productive and unproductive learning trajectories and their underlying reasons. The learning trajectories were classified according to the post-test score. To analyze the tasks from the digital workbook, machine learning methods, specifically random forest, and natural language processing, were employed to identify how students on different learning trajectories progress through the unit. The random forest analysis was accurate in distinguishing between productive and unproductive learning trajectories. Furthermore, natural language processing was employed to analyze open-ended responses, which revealed disparities in the knowledge elements that students on productive and unproductive trajectories utilized. The findings of this study indicate that machine learning techniques have the potential to provide valuable insights into student learning trajectories, which can inform the design of instructional units and the feedback provided to teachers and students.