2019 ASEE Annual Conference & Exposition

The Improving Student Experiences to Increase Student Engagement (ISE-2) grant was awarded to Texas A&M University by the National Science Foundation, through EEC-Engineering Diversity Activities (Grant No. 1648016) with the goal of increasing student engagement and retention in the College of Engineering. ISE-2 deployed a faculty development program in the Summers of 2017 and 2018, with the goal of increasing active learning, improving classroom climates, and decreasing implicit bias and deficit thinking in first- and second-year Engineering classes. In order to capture whether faculty have changed their instructional practices prior to and following the faculty development program, the ISE-2 team conducted classroom observations both before and after the faculty participants attended the trainings. To date, the team has observed 55 total classes in Spring 2017, Fall 2017, and Spring 2018, with two observers attending most of the classes.

The ISE-2 observation team consists of faculty, instructional consultants from the Center for Teaching Excellence (CTE), graduate students, and undergraduate research assistants. Thus, there was a wide range of classroom observation experiences in our team, ranging from expert to novice. In order to ensure the quality of the classroom observations and improve interrater reliability, the novice classroom observers were trained to observe undergraduate classrooms by the expert observers at the beginning of each semester. These training sessions became the basis of this classroom observation toolkit. This classroom observation toolkit can be used by instructors or researchers who are interested in doing classroom observations, but have limited experience with qualitative coding and observational research, to train themselves and their research assistants. This paper will serve as an introduction and a handbook for the classroom observation toolkit. The observation toolkit includes: a) an annotated bibliography introducing articles that would be helpful to understanding classroom observations, b) a series of training videos teaching viewers to conduct classroom observations using an environmental scan and the Classroom Observation Protocol for Undergraduate STEM (COPUS; Smith et al., 2013), one of the peer-reviewed coding schemes that is available in the engineering education literature, c) a series of sample classroom videos that document a typical first- and second-year Engineering class, d) validation keys for each of the sample videos (i.e., consensus coding of the sample videos from our team, against which a new team can compare their codes and discuss their matches and mismatches as a group), and e) guidance from the qualitative coding literature on how to determine whether emergent codes (i.e., additional codes beyond the initially adopted coding scheme) are needed and how to operationalize them.

Reference
Smith, M. K., Jones, F. H. M., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE—Life Sciences Education, 12, 618-627. DOI: 10.1187/cbe.13-08-0154.