T7-AL2-3 - Automatic Personalisation of Study Guides in Flipped Classroom: A Case Study in a Distributed Systems Course

3. Research Full Paper
Pedro Araujo1 , Nécio Veras2, Eder Farias1, Cecilia Costa1, Windson Viana1
1 Federal University of Ceará
2 Federal Institute of Ceará

In many Computer Science programs, the teaching of Distributed Systems (DS) presents challenges related to students' heterogeneous prior knowledge. Active methodologies, such as Adaptive Learning and Flipped Classroom, are used to improve the teaching-learning processes in courses sharing contexts similar to DS ones (e.g., Data Structures). The Flipped Classroom approach argues students should prepare for a class in advance by studying online material (e.g., videos, suggested reading). After, during class time, students perform problem-solving activities by using the concepts they studied before. Adaptive learning, in its turn, offers the opportunity of personalisation of the learning content based on the students’ prior knowledge and their individual learning goal, rather than the teacher determining the same course material for the whole class.

In this work, we combine these approaches aiming to offer the personalisation of study guides used in Flipped Classroom. We designed a web-tool called FCTOOL (Flipped Classroom Tool), which helps teachers to produce study guides as a Document Product Line. Teachers describe the variability of the course material and the rules to adapt it according to the students' prior knowledge. FCTOOL is integrated with the Google Suite and focuses on the class preparation time and delivery of personalised course materials to students. 

With FCTOOL, teachers create study guides (e.g., list of videos, research papers, podcasts) with obligatory and optional content. They annotate the study guide variabilities with rules related to quiz evaluations. In the next phase, the students read the study instructions and answer evaluation quizzes. With the student's responses, FCTOOL adapts the study guide offering more or less content according to the teacher's specification.  

In the classroom, teachers devote time to questions and answers about the study guide. Finally, students are involved in practices related to the content under study.

The evaluation of FCTOOL took place in three sessions during a DS course. The topics of each session were: Interoperability, Sockets and Message Queuing Telemetry (MQTT), and Blockchain. The DS class had twenty-six students with various academic levels (undergraduates and postgraduates). We evaluated the personalised study guides in two perspectives: acceptance and performance (pretest-posttest design). In each session, students answered a pretest. After that, the annotated study guide was delivered to them. Students answered the quizzes  starting the personalisation process. In the Interoperability session, FCTOOL generated seven distinct study guides. Secondly, in the classroom, students answered a posttest before performing in-class activities. In the three sessions, students increased their score significantly. For instance, in the Socket-MQTT class, they increase their average from 3.75 to 8.39 (Test t = 0.0472737, α<0.05, one-tailed). 

Moreover, from the students' perspective, the results show most students evaluated the approach positively. 83% of them agreed the study guides sent were appropriated to their prior knowledge and that the personalised course material facilitated their learning. Also, an experiment in the Blockchain session showed the personalisation process did not negatively impact students' knowledge gain. In general, our approach is a promising tool for study guide personalisation in a flipped classroom.