EXTERNAL REVIEW COMMITTEE REPORT
for
The Knut and Alice Wallenberg Foundation
July 12, 2001

Background

                  The committee reviewed the progress of The Project  to make the Conception-Design- Implementation-Operation (CDIO),  the context of Engineering Education by the Massachusetts Institute of Technology, Chalmers University of Technology, Royal Institute of Technology, and Linköping Institute of Technology during a series of meetings in Cambridge, Massachusetts, on June 26 & 27, 2001, and with materials provided to the committee prior to the meetings. The project team made a very professional set of presentations, and their written pre-read material was also most professional.

                  The review committee consists of the following four individuals. Dr. Richard M. Murray, is the Chair of the Division of Engineering and Applied Science at the California Institute of Technology. Dr. Thomas Gray is the Courses Director and former Head of the Mechanical Engineel-ing Department at the University of Strathclyde in Scotland. Dr. Kaj Holmelius is the former Senior Vice President of Research and Truck Development of Scania Trucks. And, Dr. Raymond J. Leopold, who chairs this review committee, is the Vice President and Chief Technical Officer of Motorola’s Global Telecom Solutions Sector.

Introduction

                  The committee was delighted to find that the program is well underway and has achieved an impressive set of accomplishments during its first nine months of activity. All six specific elements planned for the first year have been executed well. The cordial and effective collegiality demonstrated by the faculty and the staff of the four universities who are involved with this program is refreshing. They act as a high performance team with common goals and a genuine sense of enthusiasm for their work.

                  We strongly recommend that this program be continued.

                  At this juncture, the notion that this group of individuals can effect a new framework for engineering education may seem far fetched, but there are many parallel and complementary initiatives concerned with improving engineer education in the community at present, and the community will therefore be more receptive to a new set of ideas than in the recent past. The committee assures The Foundation that this goal is clearly within reach of this team, and the goal represents an important leap forward in engineering education curricula and pedagogy.

                  This report will summarize the committee's findings in each of the major areas of this project, and although the project plans are well founded, the committee will make some suggestions for potential improvements.

The Syllabus

                  The project team is to be commended for soliciting inputs for the syllabus from a wide spectrum of constituencies. The resulting syllabus is comprehensive and properly addresses activities in the cognitive, affective, and psychomotive domains, as described in Bloom's Taxonomy of Behaviorally-oriented Educational Objectives, and as similarly identified in other such taxonomies. The extra effort to solicit suggestions to insure the proper outline, including the relative priority of elements has yielded a high-quality product.

                  However, the survey tools and methods were created, conducted and analyzed by engineering, education, and human/machine interface professionals, and while the expansion beyond engineering is certainly commendable, it would have been wise to employ psychologists who specialize in survey methods, as well. While the results are very good, the committee cautions the project team to employ the full range of professionals in such endeavors, as there were some comments concerning the actual survey tools and the survey methods which could have been improved. (This comment on employing domain professionals should be particularly noted as the project team considers how best to address any branding and public relations activities to promote the CDIO concept).

Curriculum

                  While a high degree of curriculum design was not planned for the first year, a considerable amount of progress has been made. Prior to 1950, the major input to engineering education was by practitioners, that is, engineers with a strong industrial background. Also, a high proportion of students had a practical background of one kind or another. In the following 15 to 20 years, disciplined-based, engineering became the organizing principle, often coupled to parallel research areas. In more recent years a greater variety of approaches has been promoted (e.g., problem based learning, where specific engineering science studies are only pursued when necessary for the solution of real, holistic problems which were assigned as exercises). The innovation within CDIO evolves the organizing principle towards projects and project-based learning which is more readily preparing students to become leaders and strategic researchers. Alternative models for CDIO curricula are being employed by the four universities, and the effectiveness of each is available for analysis within this project.

                  The stated goals of motivating students to study engineering, of providing personal experiences which allow fundamentals to be more deeply understood, and of providing early exposure to system building and teamwork are well stated to provide the proper basis for the curriculum development.

                  The review committee feels the project team is well positioned to accomplish their second year goals, which will then allow curriculum changes to be included in Year Three educational programs.

Pedagogy  (Teaching and Learning)

                  The project team has very properly identified the need to evolve the pedagogic model from teaching to learning. The project team did well to survey existing methods, which have been used effectively to improve and to deepen the learning experience. They also did well to employ commercially available tools to aid active learning. They are to be commended for this demonstration of reuse and for understanding what they did not need to develop themselves. The project team convincingly demonstrated the value of active learning with meaningful metrics, hard data, and student testimonies. All four universities are employing active learning tools.

                  Most teachers, especially engineering professors and lecturers, have a clear idea of how to teach a course or deliver a lecture, but they do not necessarily think in pedagogic terms. The evidence from the project, and elsewhere, is that the students respond quickly and effectively to active learning processes, but too many teachers ignore the advantages of these methods. The project team is challenged to find better ways to impact the behavior of all the teachers, and they have demonstrated some techniques, such as team teaching, but much more needs to be done to reach the larger community of teachers.

                  The project team is also challenged to develop more effective mechanisms to link the teaching and learning initiatives to the entire CDIO curriculum, and not only to the cognitive technical topics.

Assessment

                  The project team’s vision for assessment is a system to determine the quality of teaching, learning, and program effectiveness, which can be implemented as part of a continuous improvement strategy.

                  The project team has developed a set of assessment tools, which are certainly preliminary (as planned during Year One of this project), and they need to continue to challenge whether this set of tools is sufficient to properly measure the success of their progress. And, while the project team has some knowledge, and is attempting to incorporate some form of balanced scorecard into their efforts, it remains to be incorporated in a fashion that allows a demonstrating of favorably changed behavior.

                  An opportunity exists to challenge the effectiveness of conventional, written examinations, which are a product of the earlier problem-oriented curriculum, and to create new assessment methods which help to foment a continuous improvement process which leads to better teaching, improved learning, and an overall progressive and increasingly effect engineering education program.

Workshops/Laboratories

                  There is wide agreement in academia that design-build-operate experiences are currently missing as major elements in typical engineering programs. The project team is addressing the correct topics in piloting different ways of integrating such experiences. They have made very good progress in a generating systematic approach to analyzing the needs as a step in the design process and this is a process which most engineering schools could use effectively. They have exercised this analytical process in a variety of contexts, ranging from large capital resource developments (at MIT), to projects based on minimal adaptation of existing space. Again, they have supported this work with extensive survey activity.

                  They need to give further thought to the workshops development through the use of continuous improvement initiatives.

Further Recommendation

                  This project can be successful as currently planned and as it is currently being executed; and, although the program could go on without the participation of KTH, the committee strongly feels that KTH should be funded as a full participant. KTH adds a degree of richness to the entire effort. This richness would especially be evident within the Swedish engineering academic community, and also enhance Sweden’s ability to lead in the diffusion of the CDIO curriculum.

                  The committee feels the other suggestions and recommendations the committee makes here should be considered for inclusion in the Year Three activities, and where possible considered for inclusion in the activities already planned for Year Two.

                  Previous efforts to create a framework for engineering education has grown ineffective over time, and the committee feels an effort should be made to create a living framework which can evolve over time and avoid becoming outdated. We further suggest that this be accomplished by including a Continuous Improvement element to this effort that can allow for adjustments and improvements over time. Such a program can also allow for noble contributions from among new constituencies that form as deployment of this new framework occurs across the larger academic community.

                  The current CDIO framework can currently be viewed as a closed architecture within the control and guidance of the participants in this program. The committee recommends that the project include a development of key stewardship attributes and stewardship methods, which would allow the entire architecture to evolve to an open form, which the current project team members can then steward.

                  As plans are  made to diffuse the CDIO initiative to the larger community of engineering educators, programs, and institutions, several items ought to be considered. Within the context of providing stewardship in an open architecture, methods should be developed to allow the improvements, which will come from the educators who begin to employ the CDIO concept, to be included as improvements to the living, evolving CDIO initiative. Such a capability will allow improvements from the engineering education community as a whole, and allow other entities to create a feeling of ownership, while avoiding the not-invented-here syndrome. The project team should also recognize the existence of different working cultures among the constituencies they wish to affect, and standards for allowing adoption of the CDIO initiative should be developed.

                  The efforts of the project team is heavily weighted towards the programs, students, and tools, while ways in which the behavior of the faculty can be impacted are seen as secondary considerations. Again, the review committee recommends that, perhaps under the guise of a faculty continuous improvement initiative, the way faculty behavior can be improved should be more directly included.

                  The students who addressed the review committee represented three of the four universities, and their comments were almost exclusively related to affective domain elements (e.g., motivation, stimulating, fun, realism) and people-oriented activities (e.g., interactions with professors, teamwork, communications), and seemed to pay little attention to the cognitive domain and technical activities. When the committee engaged them in conversation, they said that the technical content was of course important, but was taken as a given. The demonstrated concern by the students was obvious. What was of importance to them once inside a technical program was not the technology, since it was evident to them that the proper technical content would be there. They are concerned about their interactions with people and with peoples’ attitudes and values. This point should not be lost in the project teams activities.

Summary

                  Again, the review committee strongly recommends that this project continue to receive the support of The Foundation, and each review committee member looks forward to devoting time for the next review. The project team is to be especially commended for their fast start and collegial integration of faculty and staff in the execution of this project.

                  The potential exists here for Sweden to become the first country to widely employ the new, living, widely-accepted foundation initiative for engineering education and to be at the leading edge of its diffusion to other countries, such as the United Kingdom and the United States.