F5-O/LT3-2 - AutoNotes: Automatic Lecture Annotation Application

3. Research Full Paper
Nada Nasser¹ , Jailan Salah¹, Nada Sharaf¹, Slim Abdennadher^1
1 Media Engineering and Technology, German University in Cairo

Introduction
Full Paper
Studies have shown that technology is a very powerful tool for education. Technology integration in education can improve students' learning processes and outcomes. It also makes the teaching and learning processes more meaningful, easy and fun. Speech recognition is one of the technologies that helps in breaking down the barriers to education. The aim of the work presented in the paper is to design and implement an application (AutoNotes) that improves the students' learning outcomes by using this technology. This is done by providing the students with both auditory and visual learning and eliminating difficulties that the students face to take notes. This also facilitates the study process for the students by providing them a good and complete study material. AutoNotes is an android application that converts the speech of the instructor to text in real time during the lecture and then it synchronizes this text to the original lecture slides to produce new slides annotated with the explanation of the instructor. These new lecture slides can be displayed in a classroom during the lecture to improve the effectiveness of the learning experience of the students by giving them the opportunity to read the explanation of the instructor while listening to it. Moreover, it can be accessed by the students on their mobile phones either in the classroom or at home to study from. AutoNotes also provides a feature for the students who like to write their notes with their own way. It gives them the chance of writing notes to each slide in the lecture slides in a simple, colorful and organized way. A focus group was conducted to gather target audience opinions. The results have shown that students liked the idea of AutoNotes. They saw it beneficial for them especially if the original lecture slides are not detailed. They also like the idea of putting the notes with its corresponding slide. To evaluate AutoNotes, we have conducted a between group design setup of two identical lectures with different groups of participants. In the first lecture, the lecturer used the original lecture slides. However, in the second one the annotated lecture slides that is generated by AutoNotes was used. The participants were given several tests to compare their overall learning experiences. The results of the tests taken by the participants reveal that the participants that used AutoNotes and were exposed to the annotated lecture slides had a higher engagement level and learning gain than the other group exposed to the original lecture slides. Additionally, they had a workload level less than the other group. For the system usability test, the average usability scale that AutoNotes got is 78 which can be considered above average. Finally, the results of the accuracy testing reveal that the accuracy of the speech recognition was 83.2%. The precision and recall metrics were used to evaluate the synchronization of the speech with the slides. The average precision score was 75.6%. However, the average recall score was 51.1%.