S10-STEM5-3 - Research to Practice to Research: The Challenges of Studying Spaced Retrieval Practice in 10 STEM Courses

2. Research-to-Practice Full Paper
Campbell Bego1 , Patricia Ralston1, Keith Lyle1, Jason Immekus1
1 University of Louisville

This Full-Length Research-to-Practice paper will discuss the challenges of implementing spaced retrieval practice in a range of Science, Technology, Engineering, and Mathematics (STEM) barrier courses. Spaced retrieval practice is an evidence-based instructional technique in which multiple questions about the same topic are asked over time with delays in between. An interdisciplinary team of PIs at a large Midwestern institution were awarded an NSF grant (#BLIND) to assess the effectiveness of spaced retrieval practice in 10 different STEM courses. In the first academic year of funding, researchers held faculty workshops to develop the quiz questions that would be asked in each course. Through the interactive workshops and one-on-one reviews and conversations, researchers uncovered several challenges of implementing this technique in courses with faculty who are experts in their fields but unfamiliar with memory research.

For example, instructor participants found it difficult to write learning objectives of the appropriate grain size. Undergraduate-level STEM courses are often organized around broad learning objectives, frequently using verbs such as “solve” and “analyze.” To gain the benefits of spaced retrieval, however, students need to retrieve the same information or type of information for each question within a given learning objective. This necessitates defining learning objectives quite narrowly. Other implementation challenges included: instructors having difficulty writing sets of questions that test the same objective with comparable difficulty, and the PIs needing to learn or re-learn domain-specific knowledge outside of their expertise in order to review faculty-generated questions.

Some additional constraints and challenges were imposed by the specific nature of the research design. The research is within-subjects, meaning that each student participant will experience both massed questions and spaced questions. The design also includes counterbalancing, such that each learning objective is quizzed in a massed condition for some students and in a spaced condition for other students. Consequently, the questions that instructors generate must be appropriate for administration in either a massed or spaced condition: they cannot be the same question, because seeing the same question multiple times in quick succession in the massed condition would be strange to students, and they cannot be increasing in difficulty and depth as the semester progresses, because some students receive them all at once at an early point in the semester.

Overall, multiple rounds of communication between learning scientists and STEM experts proved to be critical. Because spaced retrieval practice has been found to improve undergraduate student recall in the short-term (i.e., within a semester) and the long-term (i.e., in the following semester), application to STEM barrier courses could improve student success, thereby potentially increasing number of STEM graduates. Identifying and describing implementation challenges will help open more classrooms to this evidence-based technique.