EVALUATION OF IMMERSIVE PROJECT-BASED LEARNING EXPERIENCES

EVALUATION OF IMMERSIVE PROJECT-BASED LEARNING EXPERIENCES

T. Rüütmann, E. Witt, T. Olowa, T. Puolitaival, M. Bragadin (2022).  EVALUATION OF IMMERSIVE PROJECT-BASED LEARNING EXPERIENCES. 313-323.

Digitalization is transforming the real estate and construction (REC) sector and a key feature of this transformation is Building Information Modelling (BIM) - the virtual representation of all building-related information. By enabling the creation of digital twins of real buildings, BIM generates opportunities to do many things in new and better ways including education and training. Specifically, BIM offers the possibility of data rich virtual environments in which project-based learning experiences can be designed. Researchers at Tallinn University of Technology, Tampere University and the University of Bologna are currently developing a prototype BIM-enabled Learning Environment (BLE) with the intention of providing more realistic, immersive and integrated learning experiences. In addition to the BLE platform itself, pilot learning modules are being created to demonstrate the potential for this approach and, to determine their effectiveness, evaluation tools are being designed. This research investigates existing, applicable evaluation models and derives an evaluation model and tools specifically adapted for the immersive project-based learning experiences provided through the BLE. A literature review was conducted to identify existing evaluation models. A comparative content analysis approach was employed to identify their specific use cases, implementation requirements, advantages and disadvantages for deployment within the BLE context. The BLE pilot learning modules were analysed in terms of their defining characteristics and the key features of evaluation models applicable to them were identified. The identified features were then integrated to derive a new evaluation model and a corresponding set of evaluation tools considering the contemporary principles of Engineering Pedagogy. The research results include: 1) Defining characteristics of the BLE pilot learning modules and the challenges these pose for evaluation. 2) Existing evaluation models and their applicability to the immersive project-based learning experiences of the BLE. 3) An outline of the evaluation model and appropriate evaluation tools for the BLE learning modules. An evaluation model together with supporting evaluation tools are proposed that will assist educators and trainers in evaluating the impact of their activities for effective engineering education. This research also serves as a guide for the development of future BLE learning modules and for evaluating their effectiveness.

Authors (New): 
Tiia Rüütmann
Emlyn Witt
Theophilus Olowa
Taija Puolitaival
Marco Bragadin
Pages: 
313-323
Affiliations: 
Tallinn University of Technology, Tallinn, Estonia
Tampere University, Tampere, Finland
University of Bologna, Bologna, Italy
Keywords: 
Engineering education
Evaluation
Project-Based Learning
Virtual environment
CDIO Standard 11
CDIO Standard 12
Year: 
2022
Reference: 
Anh, V. T. K. (2018). Evaluation Models in Educational Program: Strengths and Weaknesses. VNU Journal of Foreign Studies, 34(2), 140–150.: 
Du, J., Zou, Z., Shi, Y., & Zhao, D. (2017). Simultaneous Data Exchange between BIM and VR for Collaborative Decision Making. Congress on Computing in Civil Engineering, Proceedings, 2017-June. 1–8.: 
Ferrandiz, J., Banawi, A., & Peña, E. (2018). Evaluating the benefits of introducing “BIM” based on Revit in construction courses, without changing the course schedule. Universal Access in the Information Society, 17(3), 491–501.: 
Hwang, S., & Safa, M. (2017). Learning Advanced Decision-Making Techniques and Technologies through a Collaborative Project. In K.-Y. Lin, N. M. El-Gohary, & P. Tang (Eds.), Congress on Computing in Civil Engineering, Proceedings (Vols. 2017-June, pp. 35–42). American Society of Civil Engineers.: 
Läänemets, U., & Kalamees-Ruubel, K. (2013). The Taba-Tyler Rationales. Journal of the American Association for the Advancement of Curriculum Studies, 9(2), 1–12.: 
Lau, D. C. M. (2001). Analysing the curriculum development process: Three models. Pedagogy, Culture and Society, 9(1), 29–44.: 
Looney, J. W. (2009). Assessment and innovation in education. In OECD Education Working Papers (Vol. 24, Issue 24).: 
Nevo, D. (1983). The Conceptualization of Educational Evaluation: An Analytical Review of the Literature. 53(1), 117–128.: 
Nyathi, N. (2020). An Overview of Responsive Evaluation. https://medium.com/@nqabuthonyathim/anoverview- of-responsive-evaluation-4a7996bc3356: 
Olowa, T., Witt, E., & Lill, I. (2021). Evaluating Construction Education Interventions (M. . Auer & T. Ruutman (eds.); ICL 2020, pp. 497–508).: 
Scriven, M. (1966). Social Science Education Consortium. Publication 110, the Methodology of Evaluation. https://eric.ed.gov/?id=ED014001: 
Stufflebeam, D. L. (1983). The CIPP Model for Program Evaluation. Evaluation Models, 117–141.: 
Stufflebeam, D. L. (2003). The CIPP Model for Evaluation. In International Handbook of Educational Evaluation (pp. 31–35). Springer Netherlands.: 
Stufflebeam, D. L., & Coryn, C. L. S. (2008). Evaluation Theory, Models, and Applications. In JAMA (second, Vol. 299, Issue 22). Jossey-Bass.: 
Tranquillo, J., Kline, W. A., & Hixson, C. (2018). Student-created canvases as a way to inform decision-making in a capstone design sequence. ASEE Annual Conference and Exposition, 2018- June.: 
Walder, A. M. (2017). Pedagogical Innovation in Canadian higher education: Professors’ perspectives on its effects on teaching and learning. Studies in Educational Evaluation, 54, 71–82.: 
Witt, E., Olowa, T., & Lill, I. (2021). Teaching Project Risk Management in a BIM-Enabled Learning Environment. In Auer, M. E. & T. Rüütmann (Eds.), ICL2020 – 23rd International Conference on Interactive Collaborative Learning (AISC 1328, pp. 162–173). Springer Nature Switzerland AG 2021.: 
Go to top