This work-in-progress study began with a desire to evaluate attitudes held by engineering instructors participating in an NSF-funded IUSE project. Although instruments exist to broadly evaluate viewpoints toward student engagement (e.g., Approaches to Teaching Inventory) and to directly observe classroom interactions, few aid evaluating attitudes toward integrating specific student-centered strategies. Rather than evaluate attitudes toward expansive concepts such as constructivism or inquiry, the team focused on three concrete strategies:
1. Formative feedback
2. Integrating real-world applications
3. Facilitating student-to-student discussion
Guided by expectancy theory research, it was understood that decisions to initiate or expand the use of a classroom strategy is related to (1) perceived value for both students and self, (2) expectation of success, and (3) perceived implementation costs (e.g., time, materials). Following an established instrument development model and multiple iterations the team produced the Value, Expectancy, and Cost of Testing Educational Reforms Survey (VECTERS). In addition to course and demographic information, VECTERS contains 11 value items, 10 expectancy items, and 5 cost items. For each item, respondents provide three responses indicating dispositions on Likert scales for the three strategies listed above.
All engineering faculty members at 19 large colleges of engineering were invited via email to complete VECTERS online. A total of 286 responses were received. While it is understood that those who volunteered to complete the VECTERS may be a skewed sample of individuals interested in engineering education, the large sample and variation in responses was sufficient to test internal consistency reliability and conduct factor analysis.
The internal consistency for VECTERS items was high (Cronbach’s α = .90). Because VECTERS can be considered as three sub-instruments, comprised of 26 items separately addressing the three classroom strategies, VECTERS was accordingly examined. Cronbach’s alpha values for formative feedback, real-world applications, and student-to-student discussion (0.83, 0.76, 0.82, respectively) were all above 0.7 thus implying acceptable reliability for understanding patterns across groups.
Exploratory factor analysis was conducted using orthogonal (varimax) rotation. Analysis was applied to VECTERS' three sub-tests (formative feedback, real-world applications, and student-to-student discussion). Five to six factors were retained that were roughly parallel among all three analyses. The factors accounted for 60% (formative feedback), 55% (real-world applications), and 62% (student-to-student discussion) of the variance. Items loading heavily on the first factor for formative feedback and real-world applications, and on the second factor for student-to-student interactions, were similar; those items were mostly expectancy items indicating the strategy was expected to be successful due to student ability. For student-to-student interactions, items loading heaviest on the first factor were all value items – indicating perceived value for students when integrating the strategy. All of the cost items, except for the cost of time, comprised either the second factor (real-world applications) or third factor (formative feedback and student-to-student discussion). in sum, VECTERS has potential as a research-based tool enabling examination of rationale behind the implementation of classroom strategies.
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.
