T8-FY2-3 - Archimedes: Developing a Model of Cognition and Intelligent Learning System to Support the Development of Metacognition in Novice Programmers

1. Innovative Practice Work In Progress
Phyllis Beck1 , Jean Mohammadi-Aragh1
1 Mississippi State University

This is a Work-in-Progress Innovative Practice paper. There is an increasing need for tools and methods for gathering and analyzing data on artifacts and  understanding student behavior in introductory computing courses. Several novel approaches have been developed to support different aspects of metacognition, improve self-efficacy and provide automated feedback and assessment for students in introductory programming courses. However, these methods have not been able to successfully scale with studies that suffer from being too small or software that is abandoned due to the high overhead of developing a system to collect and analyze data on a large scale. In spite of the success of some methods, most have not been integrated into a cohesive platform where instructors can automate the data acquisition process, develop a persistent trace of time series data such as code path development and see the effects of their interventions as they are affecting their classroom in real-time. There is a need to identify what programming concepts students struggle with the most and to identify at-risk students. Early identification ensures that instructors can intervene and provide the appropriate level of support and improve a student's ability to succeed during the most formative times in their computing education. This paper introduces a multidimensional model of cognition and an application programming interface for an intelligent learning system, which we will refer to as Archimedes, that implements the first two components of the model. Our model of cognition currently includes five components thinking processes, organizational strategy, design cohesion, skill mastery, and path of program development. Together these components seek to support six strategies of metacognition. These strategies are metacognitive scaffolding, reflective prompts, self-assessment, self-questioning, self-directed learning, and graphic organizers. The Archimedes application seeks to assess introductory programming students' current level of metacognition and provide real-time feedback and allow them to self-monitor, self-assess and improve self-efficacy through the development of strategic knowledge for solving real-world programming problems. The Archimedes platform leverages state-of-the-art technology in natural language processing and machine learning to provide real-time data analytics for students and instructors. For researchers in computing education, Archimedes seeks to automate data acquisition and analysis and provide a more reliable way to accurately measure student performance and current level of mastery on introductory programming tasks. The web-based application allows us to acquire student artifacts at scale through a customizable platform that can be used to identify at-risk students and provide actionable teaching insights on classroom interventions. Additionally, Archimedes is a tool to allow researchers to conduct independent computing education research and understand what pedagogical practices are best suited for improving student success rates.