Educational Program Evaluations: Rationalizing Assessment Models and Processes for Engineering Education Quality Enhancement

Educational Program Evaluations: Rationalizing Assessment Models and Processes for Engineering Education Quality Enhancement

S. Rouvrais, C. Lassudrie, S. Ech-Chantoufi, S. Bakrim (2014).  Educational Program Evaluations: Rationalizing Assessment Models and Processes for Engineering Education Quality Enhancement. 13.

Engineering and managing educational systems remains today relatively informal, relying on shared best practices and rationales. On the one hand, in the higher education sector, international reference models permit to continuously improve the quality of educational systems (e.g. EFQM or the CDIO standard 12 for engineering education). Such frameworks are commonly used as a complement to quality systems stimulated at national level by regulation bodies or ministries (e.g. formal accreditation systems). The corresponding models incorporate a continuous improvement system, but without formal reference models per maturity levels, nor repeatable objective evaluation processes.

On the other hand, the field of software engineering has experienced a crisis several years ago and was able to cope it methodologically. It is now recognized that software quality is largely dependent on the quality of the design, development and maintenance processes. This has led many companies to align with reference model, and focus on process evaluation. As an example, the ISO 15504 (SPICE) provides a comprehensive framework for the evaluation and improvement of processes. This standard has been adapted and successfully used for specific sectors (automotive, aerospace, banking, medical, etc. ). It is based on a framework with three kind of models: (i) Capability Maturity Model, (ii) Process Reference Model, and (iii) Process Assessment Model.

The CDIO standard 12 proposes an evaluation model with a maturity scale that covers the implementation of standards, i.e. focusing on aspects of the organization, but no standardized assessment process exist, blurring its capacity to support repeatability and comparability. SPICE defines requirements on the evaluation processes, and a guide for the evaluation processes. Moreover, evaluation models include indicators for assessing process maturity: practices, inputs, outputs, resources, etc. Other frameworks like EFQM include dimensions on outcomes (performance, satisfaction, impact). The main contribution of this paper is to analyze the aforementioned quality frameworks and evaluate them in the context of higher education quality assurance, so as to propose more formalized process reference models and process assessment models to better manage educational systems in engineering,

Proceedings of the 10th International CDIO Conference, Barcelona, Spain, June 15-19 2014

Authors (New): 
Siegfried Rouvrais
Claire Lassudrie
Samia Ech-Chantoufi
Soukaina Bakrim
Institut Mines Télécom, France
Quality Assurance
Quality enhancement
reference model
assessment model
assessment process
Engineering education
CDIO Standard 12
Boele E.B., Burgler H., and Kuiper H. (2008) “Using EFQM in Higher Education: Ten years of experience with program auditing at Hanzehogeschool Groningen”. Contributions to the Higher Education Research 1: 
CDIO (2010a) “The CDIO Standards v2.0 (with customized rubrics)”. Accessed: January 2014 : 
CDIO (17 May 2010) “CDIO Standards and Rubrics with Sample Evidence”. : 
CIE - Centre for Integral Excellence (2003) “EFQM Excellence Model - Higher Education Version 2003”, Sheffield Hallam University.: 
Continuous Quality Improvement in Higher Education: A case study in Engineering School of Boras University, Ahoo Shokraiefard : 
Crawley E.F., Malmqvist J., Ostlund S., Brodeur D.R., and Edström K. (2014). “Rethinking Engineering Education: The CDIO Approach”. 2nd Edition, Springer Verlag. : 
Dounos P.K., and Bohoris G. A. (2007) “Exploring the interconnection of known TQM process improvement initiatives in Higher education with key CMMI concepts”. 10th Quality Management and Organisational Development Conference, 18-20 June, Helsingborg, Sweden. : 
EFQM (2011) “Introducing the EFQM Excellence Model”. Accessed: November 2013 : 
Gray P.J., Patil A., and Codner G. (2009) “The Background of Quality Assurance in Higher Education and Engineering Education”. In Patil, A.S. and Gray, P.J. (Editors). “Engineering Education Quality Assurance: A Global Perspective”. Springer. 316 pages. : 
ISO 9000:2005 Quality management systems -- Fundamentals and vocabulary. : 
ISO/IEC 15504-2:2003 Software engineering — Process assessment — Part 2: Performing an assessment. : 
ISO/IEC 15504-1:2004 Information technology -- Process assessment -- Part 1: Concepts and vocabulary. : 
ISO/IEC 15504-5:2005 Information technology -- Process assessment -- Part 5: An exemplar software Process assessment model. : 
ISO/IEC 14598-1:1999 Information technology -- Software product evaluation -- Part 1: General overview. : 
Kasperavičiūt R. (2011) “The application of the EFQM Excellence Model to Higher Education Schools” Mykolas Romeris University. : 
Kontio, J., Granholm, P., Valmu, H., Mäntykoski, J., Kruusamäe, K., Aukstuoliene, M., Savulionienes, L., Munkebo Hussmann, P., Edström, K. (2012) , “Supporting Programme Development with Self- and Cross-evaluations – Results from an International Quality Assurance Project”, International Conference on Engineering, pp. 816-823. : 
Lassudrie C., Rouvrais S., Kontio, J. (2013) “Managing the continuous improvement loop: A student perspective based on a semester project”. Proceedings of the 9th intl. CDIO Conference: "Engineering Leadership in Innovation and Design". 9-13 June.: 
Malmqvist J., Peterson L., and Arehag M. (2014) “Ratings of Engineering Education Program Quality – correlation between expert, alumni and program director assessments?” In electronic proceedings of the 10th Intl CDIO conference, Universitat Politècnica de Catalunya, Barcelona, Spain, June 16-19. : 
Mitasiunas, A. and Novickis, L. (2012) “Enterprise SPICE based Education Capability Maturity Model”, Workshops on Business Informatics research, Lecture Notes on Business Information Processing, Springer, vol 6, p 102-116. : 
Rouvrais, S. and Landrac, G. (2012) “Resistance to Change in Institutionalizing the CDIO Standards: From a Waterfall to an Agile Improvement Model”. 8th International CDIO Conference, Queensland University of Technology, Brisbane, July 1-4, 2012. : 
SEI (2010), “Capability Maturity Model Integration” (CMMI), V1.3. Software Engineering Institute, Carnegie Mellon University. : 
Steed, C., Maslow, D., and Mazaletskaya, A. (2005) “The EFQM Excellence Model for Deploying Quality Management: a British-Russian journey”. Higher education in Europe. pp. 307-319. : 
Go to top