Real-world engineering challenges are open-ended, multi-faceted, and exist within a societal context, requiring knowledge from multiple domains (technical, environmental, economic, and social). Undergraduates gain knowledge in each of those domains from both engineering and non-engineering classes but need guidance when drawing on that knowledge to solve new, complex problems. The research goals guiding this NSF-REE project include: (1) Identify appropriate measures of knowledge transfer/cognitive flexibility that apply to open-ended tasks; (2) Develop and adapt instructional materials and direct assessments to measure and improve students’ ability to transfer knowledge to/across problems; (3) Explore differences in students’ responses to the interventions between different types of engineering programs. The poster will share progress related to each of the goals through methods and results from two types of studies: (1) exploring neuroscience theories and measures related to cognitive load, efficiency, and flexibility for complex problem-solving; and (2) developing and testing a Sustainable Design rubric for use with multi-disciplinary engineering capstone design projects. New activity is described below, including specific objectives for the work and results.
After a successful pilot project with an electroencephalogram (EEG), we initiated a study using EEG and self-report data to investigate students’ cognitive activities when completing different tasks related to sustainability like food security and water availability. In particular we are comparing cognitive functions and activation during linear thinking (i.e., listing) tasks and systems thinking (i.e., concept mapping) tasks. We hypothesized that (1) concept maps allow individuals to organize their thoughts within a systems thinking framework, and thus result in a more complete and holistic response than a linear thinking equivalent (i.e., listing task); and (2) creating a concept map is a more complex cognitive process than writing a list of terms and thus students (at least initially) would experience greater cognitive load during cmap tasks.
Rubric work aims to provide engineering educators and students with a learning and assessment tool to enhance sustainable design outcomes of projects. In prior work, expert feedback substantiated parallel validation efforts using text mapping to establish sustainable development frameworks. Guided by the analyses, we iteratively reduced the rubric to 14 distinct criteria, and identified three scoring dimensions (quantitative & qualitative evidence, long-term thinking and lifecycle, and formal methods/documentation) as important considerations for all of the criteria rather than stand-alone criteria. The rubric development process has moved from substantive validation to structural validation with student projects. The first structural study was conducted in Spring 2018 with 51 student users for formative assessment of their preliminary capstone project work. We are currently reviewing quantitative and qualitative data to answer questions such as: (1) Is there a range of responses on the items? Are students using the full 0-3 scale? (2) Which items do we expect to be related? Which items perform as expected? (3) Is there more rating variability across individuals or criteria? The primary objective of analyzing this data, including qualitative coding, is to improve the rubric in terms of content, structure, and implementation. Future work will address other audiences and assessment purposes.
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.
