2016 ASEE Annual Conference & Exposition

Evaluating the Impacts of Different Interventions on Quality in Concept Generation

Producing ideas of high quality has great importance in engineering design. Although concept generation is sometimes one of the shorter phases of a project, concept generation that leads to viable and unique solutions can greatly contribute to a product’s final outcomes. Concept generation also has importance as a tool for engineering education and academic research. Because the quality of solutions can vary from individual to individual and from circumstance to circumstance, it would be useful to better understand how different interventions influence the outcomes of the ideation process in the concept generation stage of engineering design. In this work, we investigated the impacts of the problem context and three specific interventions designed to increase the ideation flexibility for the outcomes of concept generation. The three interventions were problem framing, design tools, and teaming. Our results show that both problem framing and teaming impact several aspects of quality, while design tools only impact the quantity of ideas produced.

This paper investigates interventions and their impact on concept generation; its main concern is which interventions affect the quality of an individual’s ideas and in what ways. The interventions under consideration include teaming, design tools, and problem framing, as well as problem context. Problem context refers to the focus of concept generation – i.e., the given design task. In this work, four unique problem contexts were studied. The three interventions – teaming, design tools, and problem framing – were created to aid the ideation process. Teaming encourages participants to share ideas as they work in teams, and design tools provide helpful design heuristics. Problem framing alters a given problem context with respect to expectations and constraints. In combination, these interventions are intended to promote ideation flexibility, one’s ability to switch between preferred and non-preferred methods of concept generation as preferred by the problem. Given insight into how the three interventions impact idea quality, engineers, educators, and students will be able to make informed decisions about which interventions to use under different conditions with different concept generation goals in mind.

Research Method: 159 engineering students from University X, University Y, and University Z were asked to participate in two sessions of concept generation. In each session, participants generated concepts and recorded their responses using design sketches and written descriptions. The first session focused on participants’ natural creative output – no interventions were applied. During the second session, participants received the aforementioned interventions in addition to the instructions used in the first session. The cognitive styles of the participants were also assessed using the Kirton Adaption-Innovation Inventory (KAI). The creative characteristics of each participant’s ideas were then measured using quality metrics taken from the literature; these included:

1. Relevance of ideas
2. Workability of ideas
3. Specificity of ideas
4. Novelty of ideas
5. Quantity of ideas
6. Variety of ideas

The scores of each participant’s multiple ideas were averaged to determine a participant’s overall performance with respect to each measure of creative output. Then, by comparing changes in these metrics across sample groups, several questions with regard to each intervention were investigated:

1. Problem Context: How does the complexity of a problem context affect creative output? Do seemingly complex or unfamiliar contexts warrant different responses?
2. Teaming: Which aspects of creative output are benefited by teaming? Do practical concerns like relevance or workability result in different outcomes than concerns for novelty or variety?
3. Problem Framing: Are individuals with certain cognitive styles more responsive to problem framing? Do preferred styles noticeably affect all measures of creativity? Or are measures like relevance or novelty more dependent on the type of problem framing?
4. Design Heuristics: How well do participants perform using design tools? Which measures of creative output are more affected by this intervention?

Using this research method, many of these questions were answered to statistical significance, and no intervention was found to have a completely positive or negative impact on creative output. For instance, adaptive problem framing was related to improvements in relevance and workability, yet it was also related to decreases in the number of ideas generated.

The complex outcomes of these interventions suggest that engineering students, educators, and professionals need to understand the inherent trade-offs of applying interventions to concept generation. When selecting interventions, individuals should first consider which aspects of creativity warrant improvement and then decide, based on that understanding, how those goals can be best achieved.