2019 ASEE Annual Conference & Exposition

Team-based projects are widely used in engineering courses [1], particularly product or process design courses in disciplines like mechanical, electrical, and civil engineering. While the intention of team-based design projects is to provide all students with a range of technical and non-technical mastery experiences [1,2], students enter into these experiences with differences - whether real or perceived – in relevant technical skills that affect division of labor amongst team members. For example, prior work by Chachra and Kilgore [3] has demonstrated a ‘confidence gap’ in open-ended problem solving between men and women specific to work on engineering design-based projects. This disparity in self-confidence leads to a lower likelihood that women will take an active role in technical tasks and instead relegate themselves to administrative or people-oriented tasks on design projects [4,5].

We hypothesize that task choice on team-based projects leads directly to learning experiences for individual students that can either reinforce or undermine self-confidence in particular engineering tasks. These effects may be particularly pronounced for underrepresented students, specifically women in traditionally male-dominated engineering fields, students of color, and first-generation students. To address this hypothesis, we developed a novel framework for synchronously quantifying students’ task-choice and self-confidence for team-based engineering design projects. This framework, which will be the focus of this study, leverages multiple validated instruments from the literature and combines them into two validated, multi-factorial outcomes – one for self-confidence and one for task-choice – that are inherently aligned with each other as well as the necessary learning elements of a typical engineering project. To facilitate visualization of individual students’ outcomes across two multi-factorial measures, we also developed a novel graphical representation of our validated task-choice and self-confidence measures.

The development of this novel framework began with the creation of holistic, validated instruments to quantify self-confidence and task-choice. For self-confidence, we developed an initial instrument by combining the well-established APPLES instrument [3,4,6], which focuses on self-confidence in interpersonal skills, problem solving, and math and science theory, with an established but unvalidated instrument [7] that measures self-confidence in tinkering and engineering applications. The combination of the two surveys allowed us to capture the entire range of typical learning outcomes of design-based projects. Exploratory and confirmatory factor analyses were conducted on data from a large pilot (N=602) of first-year engineering students. The instrument was shortened by eliminating items that did not correlate to a factor (low factor loadings) or correlated with multiple factors (high crossloadings). The refined instrument was administered to another cohort of first semester engineering students (N=632), which yielded better fit (RMSEA=.058, CFI=.857) and high reliability on each factor (Cronbach’s alpha ranging from .77 to .84). Five principal factors that encompass self-confidence or self-efficacy on design-related tasks were measured in the final version of the survey: (1) open-ended problem solving; (2) interpersonal; (3) math and science skills; (4) engineering applications; and (5) tinkering.

A similar multi-factorial instrument was developed to quantify students’ task-choice. A complete set of project tasks was constructed by identifying overlapping categories across two prior instruments [4,6] and cross-referencing these tasks with common elements found in peer evaluations, student deliverables, and grading rubrics for engineering design courses at our institution and others [8]. This yielded eight task categories: (1) problem definition; (2) concept generation; (3) prototype fabrication; (4) design schematics; (5) engineering analyses; (6) design validation; (7) project management; and (8) technical communication. Our initial task-choice survey (analgous in implementation to a prior study [6]) was administered to the same cohort of students as our self-confidence CFA. It asked students post-hoc to estimate how many hours they spent on a particular task for the entirety of the project as well as the total time that their team spent on this task. This item structure led to numerous internal inconsistencies, most notably individual time contributions exceeding total team contribution; and the item was redesigned to ask students for their self-assessed contribution relative to the team contribution for a particular task.

Our framework also includes a novel approach to graphically representing individual students’ multi-factorial outcome for self-confidence and task-choice. Influenced by the Basadaur Profile [9], a graphical representation of personal creative problem-solving strategies, we represented each students’ 5-factor self-confidence score as a radar plot, normalized to maximum achievable values. A similar approach was taken to generate a visualization of students’ 8-factor task-choice outcomes. The axes for self-confidence and task-choice were aligned by theme to facilitate direct comparison between the measures, e.g., task-choice in “project management” aligned graphically with self-confidence in “professional and interpersonal skills”. This graphical representation facilitates comparison between student mindset (self-confidence) and behavior on team-based projects.

The framework introduced in this study includes two novel elements that will be valuable in future investigations of student learning on team-based design projects. First, our measures of self-confidence and task-choice, while built on prior work [3,4,6], are the first to reflect learning outcomes and tasks that encompass the entirety of the engineering design process. In future studies, we will use these measures to gain a deeper understanding of how student behavior (task-choice) is affected by a prior self-confidence as well as demographic considerations such as race and gender. The second novel element of this work is the development of a graphical representation that aligns multi-factorial outcomes for self-confidence and self-efficacy on a per-student basis. We anticipate that this tool will be useful for both instructors and students in identifying individual students’ strengths and opportunities for growth as they engage with their peers on team-based design projects.