Assessing conceptual understanding in large engineering courses is a challenging task. When we consider that assessment in engineering education is often performed in a deterministic fashion and does not include measures of student uncertainty, the challenge is even greater. Multiple choice questions are some of the most familiar deterministic assessment instruments used in large classes. In this paper, we proposed and implemented a modified version of multiple choice response options that captures quantitative data on student self-confidence in knowing the correct answer. Using the coin distribution response method, students are given a hypothetical allotment of 100 coins that can be distributed across the answer choices for each question. The students are informed that the coin assignments should represent the likelihood that each answer choice is the correct one.
The coin distribution response method was trial-tested and evaluated in consecutive semesters of a statics course in civil and environmental engineering. During a two-week span each semester, three short quizzes were administered in this course using multiple choice questions to evaluate conceptual understanding of vectors. Results show that students develop and use different patterns to assign coins. These patterns can change from question to question. An assignment of 100 coins to a single answer choice was the most frequent response. More than 85% of those responses were correct. When students chose to distribute coins, a split of coins to two answer choices was the most common. This occurred more than twice as often as the combined occurrences of distributing coins to three and four answer choices. Collectively, about 63% of responses with coins assigned to at least two choices were correct.
Ramin is a Ph.D. candidate in Structural engineering with two Master of Science degrees in Statistics and Structural Engineering. His focus of research is uncertainty quantification in engineering problems and model updating.
Chao is a PhD student in the Department of Computer Science & Engineering at University of South Carolina. He is interested in applying machine learning algorithms and Bayesian statistics in social science study.
Dr. Caicedo is a Professor at the Department of Civil and Environmental Engineering at the University of South Carolina. His research interests are in structural dynamics, model updating, and engineering education. He received his B.S. in Civil Engineering from the Universidad del Valle in Colombia, South America, and his M.Sc. and D.Sc. from Washington University in St. Louis. Dr. Caicedo's educational interests include the development of critical thinking in undergraduate and graduate education. More information about Dr. Caicedo's research can be found online at http://sdii.ce.sc.edu
Steve McAnally is Associate Professor in the Department of Civil and Environmental Engineering at the University of South Carolina. His research interests focus on water and wastewater treatment, particularly appropriate technology applications for developing communities. Other interests include reform in undergraduate civil engineering education.
Dr. Pierce is the Director for Diversity and Inclusion and Associate Professor in the Department of Civil and Environmental Engineering at the University of South Carolina. He is a USC Connect Faculty Fellow for Integrative Learning, and a Bell South Teaching Fellow in the College of Engineering and Computing. Dr. Pierce also serves as the ASEE Campus Representative for USC.
Gabriel Terejanu has been an Assistant Professor in the Department of Computer Science and Engineering at University of South Carolina since 2012. Previously he was a Postdoctoral Fellow at the Institute for Computational Engineering and Sciences at University of Texas at Austin. He holds Ph.D. in Computer Science and Engineering from University at Buffalo. He is currently working on the development of a comprehensive uncertainty quantification framework to accelerate the scientific discovering process and decision-making under uncertainty. Some projects currently supported by NSF and VP for Research include discovery of novel catalytic materials for biorefinery industry, modeling and prediction of naturally occurring carcinogenic toxins, and development of statistical models for tracking individual student knowledge.
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.