The accuracy of RGB-D sensing has enabled many technical achievements in applications such as gamification, task recognition, as well as pedagogical applications. The ability of these sensors to track many body parts simultaneously has introduced a new data modality for analysis. By analyzing body language, this work can predict if a student will struggle in the future, and if an instructor should intervene. To accomplish this, a study is performed to determine how early (after how many seconds) does it become possible to determine if a student will struggle. A simple neural network is proposed which is used to jointly classify body language and predict
task performance. By modeling the input as both instances and sequences, a peak F Score of 0.459 was obtained, after observing a student for just two seconds. Finally, an unsupervised method yielded a model which could determine if a student would struggle after just 1 second with 59.9% accuracy.
Matthew Dering is a PhD student at Penn State University studying computer vision and deep learning.
Conrad Tucker is a professor of mechanical engineering. He focuses on the design and optimization of systems through the acquisition, integration, and mining of large scale, disparate data.
Are you a researcher? Would you like to cite this paper? Visit the ASEE document repository at peer.asee.org for more tools and easy citations.