Engineering faculty, across six disciplines, at a large university in the Southwest participated in professional development supported by an NSF Improving Undergraduate STEM Education grant during the 2016-17 academic year. The two-semester sequence involved faculty participating in bi-weekly discipline-based workshops during Fall 2016. These workshops promoted student-centered classrooms, using formative feedback to refine teaching and learning, and emphasizing real-world connections. During Spring 2017, faculty were to focus on classroom implementation while also participating in discipline-based communities-of-practice. The communities-of-practice sessions focused on themes featured in the workshops, but allowed for more give-and-take, flexibility of topics, and sharing of instructional ideas.
Throughout the academic year, classroom practices of the faculty were evaluated by trained observers using the Reformed Teaching Observation Protocol (RTOP). The RTOP is a 25-item validated observation protocol that has been utilized in numerous middle school through postsecondary projects. The RTOP focuses on gauging the degree to which learning environments are student-centered in science, engineering, and mathematics. Complete observation data were available for 26 faculty members who were observed twice early during the fall semester (pre-observations) and twice late during the spring semester (post-observations). The two pre-observations occurred during the fall semester and post-observations occurred during the last six weeks of the spring semester. It is noted that faculty were also observed twice early during the spring semester, but this study focuses on the changes between the first and last sets of observations.
Because many faculty members became interested in the use of the RTOP and were eager to improve their instructional approaches, an unplanned strategy to support faculty emerged. Faculty members (n = 21) requested to receive one-on-one feedback with an observer who was also an experienced K-12 instructional coach. These coaching sessions typically honed in on a few RTOP items, provided the instructor with insight about student engagement, and offered concrete suggestions for improving instructional practices.
The purpose of this study was to determine the effects of (1) participation in the professional development, and (2) receiving coaching, on instructional practices, as measured by the RTOP. Analysis indicated RTOP mean scores increased significantly (n = 26, p = .014) from pre-observations (x̄ = 57.1) to post-observations (x̄ = 68.6).
The comparison of the small group of five faculty members, who opted to not receive coaching, to the 21 faculty members, who opted for coaching, did not allow for robust comparison. However, although not statistically significant, the coached group had notably greater mean RTOP gains from pre-to-post (Δ = 12.8, SD = 12.8) than the non-coached group (Δ = 6.2, SD = 8.8). Normalized gain scores (aka Hake scores) were calculated to provide comparisons independent of pre-RTOP scores. The normalized gain scores are calculated as a proportion of change in RTOP scores compared to possible change. Here too, although not statistically significant, there were indications of the benefits of being coached. The coached group had conspicuously greater normalized gain scores (g = 0.20) than the non-coached group (g = 0.13).
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.
