S10-O/LT8-6 - Tracking Representational Flexibility Development through Speech Data Mining

3. Research Work In Progress
Fengfeng Ke¹ , Jewoong Moon¹, Zlatko Sokolikj^1
1 Florida State University

Introduction
This work-in-progress presentation aims to report an empirical study examining the development of representational flexibility for adolesents, via real-time data mining and modeling of the speech data from learners who were involved in scientific simulation making. Representational flexibility—the ability to consider simultaneously multiple representations of a single object or event and to switch flexibly between them based on a changed condition—is a critical index for computational thinking and practices (Cheng, 1999; Duval, 2006; Nistal et al., 2009; Spiro, 1988; Thomas, 2008). Representational inflexibility can and should be dynamically fostered in a diverse student population to ensure their talent for engineering and computing fields does not go untapped.

In the current research we exploited and investigated a virtual reality (VR) based, flexibility learning environment (FLE) in which adolescents with autism use, customize, and design an assortment of simulation games that represent and exemplify the application of forces and Newton’s laws of motion. The simulation/game modding and making tasks acted as the primers of practicing and assessing representational flexibility (RF) in solving the problems of forces and Newton’s laws of motion.  The study to be reported addresses the following research questions:

1. What is the applicability of using speech data mining as a performance assessment method of representational flexibility?
2. To what extent will learners with and without ASD develop representational flexibility during scientific simulation design and coding based on speech data mining?

Eight adolescents with and without autism spectrum disorders (aged 14-19) participated in the FLE program thatconsists 12 one-hour intervention sessions, with a staggered start of the intervention sessions among them based on the multiple baseline across-subject research design. The baseline data and repeated measures of RF during the program were collected via a packet of representational flexibility tasks, including the Wisconsin Card Sorting Test, spatial and mathematical insight problems, and VR simulation-based artifact design problems.

The participants’ participation behaviors and verbal utterances during the intervention sessions have been archived via screen and webcam recordings. We implemented two educational data mining approaches to assess the participants’ representational flexibility competency states with the transcribed speech data: (1) developing a multi-label, competency-classification model and (2) using the similarity index. 

The current study findings indicated that two approaches of speech or text data mining, multi-label classification and similarity index, can act as the in-situ performance assessment methods to evaluate the representational flexibility development of a heterogeneous learner group. These methods, used along with free and open speech-to-text and data mining tools, can afford a longitudinal and non-intrusive tracking of cognitive flexibility states during scientific problem solving. The findings also suggest that VR-supported, design-oriented simulation and game design tasks can act as primers for representational flexibility acquisition and assessment. The findings on the flexibility learning environment will help to advance the research and training of representational flexibility for problem solving in engineering and computing education.