S7-BR7-2 - Evaluating the validity and reliability of Approaches to Teaching Inventory (ATI) in higher engineering education research and development (WIP)

3. Research Work In Progress
Johanna Naukkarinen1
1 Lappeenranta-Lahti University of Technology LUT

This abstract describes research work in progress. Approach to teaching inventory (ATI) [1] is a widely used tool in higher education research and development. It has been used, for example, to measure and explain teachers’ repertoire of teaching methods [2], to evaluate the impact of pedagogical training on teachers’ conceptions [3], to study the development of teacher identity [4], and to understand the variety of teaching conceptions [5]. The instrument was originally developed in the contexts of science teaching, but it has been applied in different disciplinary contexts, and used to understand the disciplinary variation in teachers’ thinking [5, 6, 7]. Yet, the applicability of the instrument in engineering education remains somewhat unclear. The results regarding disciplinary variation are contradictory, with some studies showing evidence on disciplinary differences [6, 7] and some not [8]. In addition, many of these studies included no engineering teachers or the studied disciplines were clustered in such a way that it was not clear whether engineering was among the disciplines under study.

In the disciplinary variation studies, engineering is usually defined as a hard applied science [9]. Lueddeke [6] suggests that “staff teaching hard/pure or applied subjects are more likely to bring an ITTF orientation to their teaching, while staff teaching soft/pure or applied subjects generally take a more developmental (constructivist) approach in classroom situations (i.e., CCSF)”. Yet, the mean Conceptual Change/Student Focus (CCSF) score of technology teachers was higher than the mean Information Transmission/Teacher Focus (ITTF) score. Nevgi et al. [7] discovered disciplinary differences between hard and soft disciplines, but no differences between pure and applied disciplines. In their data, there was no statistically significant difference between the ITTF and CCSF scores of teachers from hard disciplines, whereas the teachers from soft disciplines were significantly more student than teacher focused. This study, however, included no engineering teachers.

Recent studies using ATI and focusing solely on engineering teachers suggest that the engineering faculty have stronger CCSF than ITTF orientations [10, 11]. When conducting one of these studies, the wording of the ATI statements and especially the leanings in different forms of discussion as an indication of student-centeredness drew the researchers’ attention and raised questions about the validity and applicability of the instrument in the engineering discipline. This generated a research idea of systematically examining both the validity and reliability of ATI in engineering. The study of reliability will be conducted statistically based on the data combined from [10] and [11], which will be supplemented with some new data. The content validity of the instrument will be studied qualitatively, and in this quest, suggestions and insights of the FIE community would be of great value.

Keywords: Approaches to teaching inventory (ATI), content validity, instrument reliability


[1] Trigwell & Prosser, 2004.
[2] Coffey & Gibbs, 2002.
[3] Postareff et al., 2007.
[4] Nevgi & Löfström, 2015.
[5] Stes & Van Petegem, 2014.
[6] Lueddeke, 2003.
[7] Nevgi et al. 2009.
[8] Stes et al., 2010.
[9] Becher, 1994.
[10] Niiranen & Naukkarinen, 2019.
[11] Beagon & Bowe, 2019.