Graduate student James Shetter will be defending his thesis titled “Using Artificial Intelligence and Machine Learning Techniques to Predict Design Process and Approaches.”

James Shetter thesis defense

• Date: Friday, April 24
• Time: 1-2 p.m.
• Location: BARC 2221
• Current Major: M.S. mechanical engineering
• Thesis Committee Chair: Dr. Apurva Patel
• Committee Members: Dr. Elisabeth Kames, Dr. Alexander Murphy, and Dr. Ala Alnaser.

Abstract

Understanding how engineering students design processes and problem-solving approaches is crucial for enhancing engineering education, developing problem-solving skills, and fostering the creation of effective design teams. This study uses a data-driven approach coupled with surveys to predict students’ approaches to design and problem-solving.

A multiphase methodology was developed and implemented using survey instruments to capture a cognitive profile on students that include measures such as learning preferences, self-efficacy, and cognitive style. Data was collected from a sophomore-level design-focused course from a STEM-focused university. These survey responses were encoded into numerical features and used as inputs in a variety of machine learning (ML) models. The two outcome variables of this study were an iteration score, representing the student’s willingness to iterate, and a bias score, showing if the student preferred front-end design or back-end design. Both regression and classification models were evaluated using cross validation, and feature selection methods were applied to identify the most informative predictors. The accuracy of the predictive model is evaluated by analyzing deviations between expected and actual design behaviors, providing insights into the effectiveness of pre-survey-based predictions.

Results indicate that predictive performance across all modeling families was moderate. Regression models exhibited low exploratory power, with near zero R2 values, while classification models achieved modest accuracy results, generally below 50% under a three-class formulation. A sensitivity analysis revealed that performance improved when classification problem was reduced to two class formulations, suggesting that design behaviors are broader. Feature selection methods consistently showed a small subset of variables, such as confidence and risk propensity, as the most informative predictors; however, these features alone were not sufficient to produce strong predictive performance.

These findings suggest that while ML techniques can identify general patterns in design behavior, pre-survey cognitive and affective measures provide limited predictive power. These results also highlight the complexity of modeling design behavior and suggest that different or additional data points may be required to improve accuracy. This work contributes to the growing body of research on ML in engineering education by evaluating the feasibility and limitations of predicting design approaches prior to task engagement.

Biography: James Shetter is a master’s student in mechanical engineering at Florida Polytechnic University, where his research focuses on engineering education, design cognition, and the application of machine learning to understand and predict student design behaviors. His work centers on developing predictive frameworks that leverage survey-based constructs, such as cognitive style, self-efficacy, and risk propensity, to model how students approach and execute engineering design tasks.