CHATGPT AS A CO-PILOT FOR ASSESSMENT DESIGN REFINEMENT: AN EXPLORATORY STUDY

The evaluation of students' attainment of course learning outcomes is a fundamental aspect of a successful engineering program, exemplified by CDIO Standard 11 'Learning Assessment'. However, earlier research has identified a prevailing gap in assessment competency among faculty. Rooted in established interdisciplinary concepts and theories, this study aims to explore the usage of ChatGPT-4 as a co-pilot to guide faculty in assessment design refinement. To achieve this goal, we adopt a conversational analysis approach, contextualizing our study within the settings of the final exam of the senior course “Wireless Sensor Networks”, offered at ESPRIT School of Engineering. We propose a framework to guide the implementation of the conversational analysis method. Our research results illustrate the merits, potentials, and limitations of using ChatGPT as a co-pilot to assist faculty in refining the assessment design process. It also brings into evidence the importance of keeping a ‘human in the loop’ perspective during the faculty-ChatGPT assessment co-creation activities. Our study can pave the way for further research on other potential applications of “Human-AI co-creation” and augmented man-machine intelligence in a CDIO engineering education.