Motivation and Background
Co-curricular engineering experiences range from very informal, such as face-to-face social networking within an engineering club to service learning experiences that approximate the workplace. Such experiences situate learning within an environment that may foster integration of knowledge and skills to solve problems (Pierrakos, Borrego, & Lo, 2007). These informal learning environments represent degrees of complexity. Therefore, applying design or problem solving within such environments may also lead to other types of latent outcomes, such as innovative ideas or adaptive expertise, or navigational flexibility (Author, 2014a; Pellegrino, DiBello, & Brophy, 2014; Sawyer & Greeno, 2008; Stevens et al., 2008). However, research about how to assess outcomes attained via participation in informal learning endeavors is nascent. We applied situated learning theory as a theoretical framework (Author, 2014b). We empirically examined student outcomes across six different informal or co-curricular experiences, as aligned with the 11 ABET (2013) student outcomes, outcomes that are critical to students learning to become an engineer (Balascio, 2014). While we report out the results based on the ABET outcomes, the main purposes for this paper were to: (a) examine the nature of the reflective discourse that revealed student self-directed learning of non-cognitive outcomes; (b) investigate the interactive nature of the technical and professional skills outcomes; and (c) explain the methods used to map the reflective discourse to develop assessments of student outcomes.
Methods
We used qualitative methods to collect and analyze the data. Then, we applied content and response process validation methods (AERA, APA, NCME, 2014; Messick, 1995). This study took place in two large public universities. Both with multiple engineering majors. Data were collected from undergraduate engineering students participating in several different informal environments, including a engineering competitions, service learning, and a robotics club. Approximately 60 students participated across the events. We conducted focus groups and individual interviews with students participating in the challenge. Focus groups provided insight into the collective learning experiences (Ryan, Gandha, Culbertson, & Carlson, 2014). Sample protocol included open-ended questions, such as: “How would you describe your process?”
Results
Preliminary Results indicated the incidence of learning outcomes associated with each of the ABET (2013) student technical skills. In addition, outcomes were inter-related across the ABET criteria. For example, students’ reflections often integrated their learning processes about design, experimentation, and problem-solving. Importantly, students also demonstrated adaptability throughout the competition. Illustrative examples of the mapping of response processes and test content are provided.
Implications
This study provided evidence of student learning across the ABET technical and professional skills across a range of informal and co-curricular engineering experiences adding to the growing body of research examining learning in informal environments. More importantly, students engaged adaptive expertise (Walker, Cordray, King, & Brophy, 2006) understanding the need for flexibility and self-evaluation to solve a problem (Zimmerman, 2002). In addition, the study provided examined response processes as a validation method to establish test content.
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.
