Authors: J. Schlenger, T. Yeung, S. Vilgertshofer, J. Martinez, R. Sacks, A. Borrmann
University: Technical University of Munich, Germany, Israel Institute of Technology, Israel

Lean construction, originating from lean management, aims to proactively improve the overall efficiency of construction processes. This requires continuous assessment of performance to identify key problems, which allows continuous improvement of construction processes. Novel digital twin approaches form an excellent technological foundation for performance assessment through regular updates with product and process information directly from the construction site. While some publications favour a schema-less approach to digital twins, we argue that well-defined data structures are required to represent complex information reliably and transparently. Existing process and product models are inadequate with respect to the requirements of a digital twin of the construction phase. As a result, we introduce a new process-oriented model that provides an improved basis for advanced process evaluation in the digital twin environment. This data schema, which is the main outcome of this paper, is presented in UML format together with a first approach to transfer it to an ontology usable in the Semantic Web context.

Authors: André Borrmann, Jonas Schlenger, Nicolas Bus, Rafael Sacks
University: Chair of Computational Modeling and Simulation, Technical University of Munich, Germany; Centre Scientifique et Technique du Bâtiment, Sophia-Antipolis, France; Technion – Israel Institute of Technology, Haifa, Israel

With the increasing adoption of the Digital Twin concept in the construction industry in the operations and maintenance phase, researchers and practitioners are increasingly seeking suitable technological solutions for the design and construction phases. While it is widely accepted that the required platforms hosting the digital twin must be cloud-based to fulfill the requirements of ubiquitous accessibility and centralized consistency, questions regarding the need for data schema remain. Some academics argue that a structure-free organization of data is suitable for realizing digital twins and the data streams from and to the respective platform. Hands-on experience in the BIM2TWIN project supports a counter argument, i.e., that structure-free data is insufficient for most use cases around AEC Digital Twins. The sheer information complexity of construction projects requires well-defined data structures enabling unambiguous and errorless interpretation. This becomes apparent when reflecting on the well-established concept of the data-information-knowledge pyramid describing that raw data must be processed into understandable and meaningful high-level information for human decision makers, subsequently providing the basis for crossproject domain knowledge. Based on this observation, we highlight that objectoriented modeling is a widely recognized information modeling technique that facilitates the structuring of complex domain information. We compare it with ontology-based model concepts that provide a similar, yet more abstract means for information modeling.

Authors: Johansen, K.W., Nielsen, R.O., Teizer, J. , Schultz, C.

Authors: Fabian Pfitzner, Jonas Schlenger and André Borrmann
University: Technical University of Munich, Germany

Site schedules are usually developed by the rule of thumb based on the experience of on-site managers. While this approach can be suitable for smaller job sites, it is challenging to make good decisions for larger projects. Planning errors can result in massive delays and increasing costs. Significant improvements in other industries showed that data-driven productivity analysis of past processes advances the planning and execution of current and future projects. However, in the Architecture, Engineering & Construction (AEC) domain, automated productivity analysis of the construction phase has barely been investigated. To overcome this deficiency, this paper presents a first approach for multi-level productivity analysis of shell constructions. We discuss several state-of-the-art vision-based technologies that serve as a foundation for large-scale evaluation of the progress on a construction site. A complete pipeline is introduced that uses different types of neural networks to extract productivity information from images at various levels of detail. The proposed workflow is demonstrated for the construction process of cast-in-place concrete pillars, implementing the first two layers. Finally, remaining challenges are discussed.

Authors: Timson Yeung, Jhonattan Martinez, Li-Or Sharoni, Jorge Leao and Rafael Sacks
Published by: IDP Ingeniería, Seskin Virtual Construction Laboratory-Technion

In construction, simulation can provide production planners with forward looking or predictive situational awareness of the potential impact of proposed changes before implementation. Planners can experiment extensively with various alternative production plans and systems without suffering real-world consequences of failure. Addressing the need to have proper control of the jobsite, DTC is a model for managing production in construction that leverages data streaming from different monitoring technologies and artificially intelligent functions. Overall, DTC offers accurate project status information (PSI) and proactive analysis and optimization of ongoing design, planning, and production processes. The integration of automated monitoring and information integration algorithms contemplated within the DTC framework may be able to provide the kind of information needed for practical simulation at short intervals, thus offering construction planners a powerful tool to optimize the decision-making process regarding any necessary changes to designs or plans, by automatically generating accurate and reliable simulation models based on the current jobsite progress, resource information, and safety conditions. This paper describes an automated system for parametric generation of simulation models for this purpose from project intent and status information stored in a DTC database. This is one aspect of broader research that involves design, development and testing of a DTC simulation and optimization system. A construction case study is provided to demonstrate the technical feasibility of automatically and parametrically producing simulation models based on data from a digital twin.

Authors: Karsten W. Johansen, Rui Pimentel de Figueiredo, Olga Golovina and Jochen Teizer
University: Civil and Architectural Engineering, Aarhus University, Denmark – Electrical and Computer Engineering, Aarhus University, Denmark

Construction sites are dynamic, and the environment is changing fast, which means the collective safety equipment, such as fall protection barriers, should also be changed to keep it compliant with the construction codes. However, the safety equipment can become non-compliant for several reasons, e.g., temporal removal in combination with incorrect or omitted re-installation or changes in the building. Thus, there is a demand for frequent inspection of the equipment, which is time- and labor-intensive as this is currently done through manual examination by safety experts. In this work, we utilize an unmanned aerial vehicle (UAV) to detect the presence, absence, and defects of safety equipment in construction work-site environments. Furthermore, the UAV continuously inspects and provides safety object location information that human collaborators can use to improve safety within the environment. We utilize an 3D occupancy grid representation to map the environment and compact point pair feature representations for efficient and robust object recognition and pose estimation. To assess the applicability and accuracy of our methods for model-based pose estimation of BIM structures, we created a realistic simulation construction environment. A set of experiments demonstrate the applicability and precision of drone-aided localization and inspection of safety equipment in the construction industries.

Authors: Johansen K.W., Schultz C., Teizer J.
University: Department of Electrical and Computer Engineering, Aarhus University, Denmark –  Department of Civil and Mechanical Engineering, Technical University of Denmark, Denmark

Due to the continuous changes in its complex and dynamic work environment, work in the construction industry is one of the most dangerous. Many of the existing workplace safety planning techniques are still based on 2D drawings and manual expertise. This effort is cumbersome as the progressing work quickly results in outdated safety plans. Researchers have put much effort into  automating  the  planning  process,  but  their  result’s  soundness  and  completeness  are incomparable. This work describes our BIM-based ontology of construction hazards and mitigation interventions for fall from height hazards based on EU and US regulations. We extract the variations of the rules and capture the concepts in spatial artifacts. We carefully created a benchmark model that  allows  for  soundness  and  correctness  assessment  which  enables  comparison  of  different automated PTD approaches.

Authors: Maciej Trzeciak, Kacper Pluta, Yasmin Fathy, Lucio Alcalde, Stanley Chee, Antony Bromley, Ioannis Brilakis, Pierre Alliez
University: Department of Engineering, University of Cambridge, Inria Sophia Antipolis-Méditerranée, 3 Laing O’Rourke

Hand-held scanners are progressively adopted to workflows on construction sites. Yet, they suffer from accuracy problems, preventing them from deployment for demanding use cases. In this paper, we present a real-world dataset collected periodically on a construction site to measure the accuracy of SLAM algorithms that mobile scanners utilize. The dataset contains time-synchronised and spatially registered images and LiDAR scans, inertial data and professional ground-truth scans. To the best of our knowledge, this is the first publicly available dataset which reflects the periodic need of scanning construction sites with the aim of accurate progress monitoring using a hand-held scanner.

Authors: Jiabin Wu and Jonas Schlenger
University: Technical University of Munich, Germany

Building designing is an iterative process of developing design concepts while fulfilling various requirements. Design parameters, dependencies, and constraints are embedded in the BIMbased design environment to support automated design adaptation techniques. However, only a small part of design constraints is explicitly represented in the digital models as design and engineering knowledge, and most studies focus on constraints on single object levels. To address this issue, this paper presents a workflow for enriching building knowledge graphs with design-oriented constraints. This research aims to extract constraints through embedded design parameters automatically. Data retrieval queries and analyses for model constraints are accomplished based on the extracted RDF graph that represents the intended building topology. Maintaining the users’ design intent and obeying the consistency constraints, the graph-based approach dynamically computes the range of design parameters potentially associated with the requirement constraint fulfillment. Due to the graph structure, cascading effects of element displacements can be considered on various levels of adjacency.

Authors: Viktor Drobnyi, Yasmin Fathy and Ioannis Brilakis
University: University of Cambridge, Cambridge, UK

Generation of geometric Digital Twins of existing buildings relies on point cloud datasets and is still a manualintensive and time-consuming process. This paper identifies the most frequent object types in buildings, analyses how current commercial software and state-of-the-art research methods to generate geometry of these objects from Point Clouds. We summarise the main advantages of these methods and highlight limitations that limit these methods from broader adoption by the industry. Later, we identify the open challenges and discuss future directions to enable automating geometric Digital Twin generation.

Authors: Zhiqi Hu, Yasmin Fathy, Ioannis Brilakis
University: University of Cambridge, Cambridge, UK

Geometry updating for digital twins of buildings is a timeconsuming and manual task, resulting in poor progress monitoring and quality control during the construction stage. This paper reviews the state of the art in practice and research on spatial and visual data-based approaches for updating digital twin geometry of buildings. We draw novel key insights into the effectiveness, experiments, and limitations of seven classes of methods summarised from the most recent papers. Consequently, four core gaps in knowledge are investigated. Finally, a new geometrybased object class hierarchy is derived to support geometry updating for maintaining digital twins in future directions.

Authors: Zhiqi Hu, Ioannis Brilakis
University:Department of Engineering, University of Cambridge, UK

The lack of timely progress monitoring and quality control contributes to cost-escalation, lowering of productivity, and broadly poor project performance. This paper addressed the challenge of high-precision structural instance segmentation from point clouds by leveraging as-designed IFC models in Scan-vs-BIM contexts. We proposed an automatic method to segment the entire points corresponding to the as-designed instance. The workflow contains: 1) Instance descriptor generation; 2) PROSAC-based shape detection; 3) DBSCAN-based cluster optimization. The method matches design-intent planar, curved, and linear structural instances in complex scenarios including: 1) the as-built point cloud is noisy with high occlusions and clutter; 2) deviations between as-built instances and as-designed models in terms of position, orientation, and scale; 3) both Manhattan-World and non-Manhattan-World instances. The experimental results from five diverse real-world datasets showed excellent performance with mPrecision 0.962, mRecall 0.934, and mIoU 0.914. Benchmarking against state-of-the-art methods showed that the proposed method outperforms all existing ones.

Authors: Several Authors
University: SPHERE Project with the collaboration of BIM2TWIN, BIMERR, BIM4EEB, BIMPROVE & COGITO projects

A review of existing ontologies in construction is presented. Motivation, alignments and final applications with special interest on methodologies. Software Tools are presented in an Appendix. Some use cases are presented with different orientations. Final conclusions are oriented to new paths to follow in future developments, considering other technologies and commercial solutions nowadays. More than new ontologies the use of existing ontologies as DICon1 could help to standardize and make an ontologies map for construction possible. Progress with standards, as mentioned in a specific chapter about status of group CEN442 WG4, is essential to get a good level of interoperability, but it faces complex technical challenges. And the most sophisticated and perfect implementation sometimes means a non-practical approach, and too rigid to be used by the industry.

Authors: Agnieszka Mikołajczyk, Raúl García-Castro, Rahul Tomar, Rehan Khan,Wojciech Teclaw, Bruno Fies, Jonas Schlenger
University: Technical University of Munich, Germany; Centre Scientifique et Technique du Bâtiment, Marne-la-Vallée, France; SINTEF, Trondheim, Norway; DigitalTwin Technology GmbH, Koln, Germany; Universidad Politécnica de Madrid, Madrid, Spain; ASM Research Solution Strategy, Kutno, Poland

This article is a result of joint workshop which took place during Sustainable Places 2022 conference in Niece, France, and was coorganized by the Building Digital Twin Association (BDTA) and six EUfunded projects that have developed a construction-phase digital-twin data model, and their ontological representation, which is capable of capturing all data requirements for the digital representation of building and/or infrastructure construction sites. Four of the EUfunded projects participating in the event contribute to this Open Letter which aims to highlight the relevance of ontology in the digital twin environment, and the approach by the different EU-funded projects. All four LC-008-EEB funded projects contributing to this article (BIMprove, COGITO, ASHVIN and BIM2TWIN), agreed on joining forces for raising awareness around Digital Building Twins and its impact in the construction industry. Their primary aim is to share knowledge, experiences and research outcomes with other stakeholders and communities around the EU and beyond, via online communication like webinars, newsletters, social media channels and scientific or technical articles. This initiative aims at delivering the wide range of digital tools for the construction sector needed on the European market and to raise awareness about the benefits coming from their use.

Authors: Timson Yeung, Jhonattan Guillermo Martinez Ribón, Li-Or Sharoni, Rafael Sacks, Tomi Pitkäranta
University: Seskin Virtual Construction Laboratory, Technion – IIT, Haifa, Israel; Sitedrive Oy, Helsinki, Finland

Production system design, planning and control are limited both by the incomplete situational awareness of planners and by their inability to predict the range of possible outcomes of their planning and control decisions. With the development of information technologies for monitoring products and processes on construction sites, it is increasingly possible to provide detailed status information describing the as-built products ‘as-built’ and processes ‘asperformed’. This opens the door to applying predictive analytics to provide decision-makers with frequent predictions of the outcomes for a range of changes they might contemplate to the production system design, even during construction. Within the BIM2TWIN project, we are designing and implementing an agent-based simulation engine that is a core component of an Automated Decision Support System. Currently, the simulation can be calibrated to accurately predict the range of likely project durations for a residential construction project. However, certain aspects of the trade crews’ performance, particularly with respect to the completion of tasks, appear to differ from the behaviours described by industry experts and encapsulated in the crew agent behaviour tree in the simulation.

Authors: Schlenger J., Vilgertshofer S., and Borrmann A.
University: Technical University of Munich, Germany

Automated progress monitoring builds an important foundation for objective productivity analysis of construction processes. Digital twins of the construction phase rely on fully automated approaches to acquire near real-time progress information. This is essential for identifying bottlenecks during construction and supporting future project planning. Many existing vision-based methods lack automated image acquisition, fast computation times, or fine-grained progress information. This paper presents a new vision-based construction monitoring approach that reduces the geometric information provided in exchange for a higher time resolution and a higher level of automation. Instead of the detailed geometry, the real-time status of the building elements is provided. It is applied to cast-in-place concrete columns, identifying individual operational steps. The approach is based on projecting building elements from a building model onto images of a fixed on-site camera to then classify them according to the current element status with the help of a CNN. Using image sequences additionally allows accounting for moving objects and other outliers, which makes the approach robust and reliable.

Authors: Jhonattan G. Martinez, Timson Yeung, Rafael Sacks
University: Virtual Construction Laboratory (VClab), National Building Research Institute, Technion–Israel Institute of Technology, Haifa 3200003, Israel

Production planning in construction requires numerous agents, or production planners, to make operational decisions that affect the project outcomes. Decisions are based on information collected on the job site and in the project supply chain. However, difficulties accessing real-time information, the numerous production planners involved, and limitations on the planners’ degree of freedom of action can hinder decision-making. Digital Twin Construction (DTC) has emerged as a paradigm for systems that increase the situational awareness of the construction project among production planners and reduce uncertainty in the decision-making process. Under the DTC frame, this research seeks to determine the scope of action of production planners. To accomplish the research goal, semi-structured interviews and a literature review were carried out to identify production planners’ degree of freedom in decision-making when faced with the need for product or process changes. First, the research identifies the operational decisions that production planners make during a construction project in response to developments (as-built, as-performed information). Second, it presents a detailed analysis of production planners’ main limitations during decision-making. Finally, the freedom of action of production planners was defined according to their roles. The findings are summarized in a matrix that associates operational decisions, degree of freedom, and professional roles in the context of DTC. The study results showed that production planners’ scope of action is limited by the lack of real-time information concerning the construction project status and technical and legal limitations that affect their decision-making process.

Compared to other industry sectors, the construction sector’s productivity is relatively low. By collecting data directly from the construction site, the main bottlenecks can be identified and support decision-makers in making well-informed decisions. This is by no means a simple task because of the complexity and unpredictability of construction sites and the challenges of monitoring heterogeneous on-site data. While other researchers focus on elaborating specific steps or use-cases of the construction monitoring processes, we present a holistic workflow for scalable shell construction monitoring that uses state-of-the-art data processing techniques. The result of the proposed workflow is a per-instance database for on-site progress across time. Such a database has many possibilities for application. This can range from giving an overview to construction managers, providing a backbone for sophisticated analysis and digital twins, to the validation of computer vision approaches.

Authors: Jhonattan G. Martinez, Timson Yeung, Rafael Sacks, M. ASCE, Yasuer Shahaf and Li-Or Sharoni
Published by: Technion–Israel Institute of Technology, WePromotion

Production planners in construction face several problems related to the lack of information regarding construction project status, changes to resource availability, instability in production rates, design changes, and limited situational awareness (SA). Poor SA leads to uninformed and suboptimal resource allocation decisions. As a result, the construction workflow is negatively affected, degrading project performance. This study focuses specifically on possible relationships between production planners’ SA and the resulting workflow quality. Such understanding can lead to the development of measures to improve production management in construction. Interviews with diverse production planners highlighted the nature of the information needed, issues related to the lack of SA, and the effect on workflow. A serious roleplaying game was devised to measure the correlation between production planners’ SA and workflow, and 14 live simulation experiments were conducted with two teams of nine players each. Finally, a qualitative analysis of the information flow configurations applied in the experiments evaluated the expected effects of information flows in the live experiment. The study found a positive correlation between the quality of information flow and workflow in construction, indicating causation in line with theoretical expectations. Specifically, free communication among planners together with management tools such as formal weekly work planning and a digital planning tool reduced the assembly time, working time, and non-value-added time of the subcontractors in the experiments. Good information flows among production planners and an up-to-date jobsite status diminish uncertainty and increase SA, which lead to improved workflow and productivity.

Authors: J. Teizer; K. W. Johansen; and C. Schultz
University: Dept. of Civil and Architectural Engineering – Dept. of Electrical and Computer Engineering, Aarhus Univ., Denmark

“Digital twins” as models for information-driven management and control of physical systems have emerged over the past years in multiple industrial sectors and recently also in construction. However, in the domain of construction safety, a digital twin remains undefined, with little or no consensus among researchers and practitioners of two essential aspects: (1) the connection between the physical reality of a construction site (the “physical” twin) and the corresponding computer model (the “digital” twin), and (2) the most effective selection and exploitation of real-life data for supporting safe design, planning, and execution of construction. This paper outlines the concept for a Digital Twin for Construction Safety (DTCS), defining three essential steps in the digital twin workflow: (1) safe design and planning for hazard prevention, (2) risk monitoring and control for proactive prediction and warning, and (3) continuous performance improvement for personalized- or project-based learning. DTCS should be viewed as a system-based approach enhancing the overall safety performance rather than exclusively integrating sensing information or safety knowledge in Building Information Modeling (BIM) for safety purposes. The result is an outline of our vision of the DTCS and a description of its components. Additionally, we point toward future research on the topic.

Authors: T. Yeung, J. Martinez, L. Sharoni, R. Sacks 
University: Virtual Construction Laboratory, Technion – Israel Institute of Technology, Israel

Fostered by advances in digital twin and onsite monitoring technologies, the paradigm of Digital Twin Construction (DTC) has emerged as a comprehensive mode of construction that proposes a data-driven lean production planning and control workflow, leveraging project status and design intent information to make proactive production system design changes. Simulation plays an essential role in the DTC paradigm as a provider of predictive situational awareness (SA), a mechanism for data-driven continuous improvement, and an enabler of future autonomous real-time production control systems in construction projects. In this paper, we outline these modes of use of simulation, discuss potential barriers to implementing simulation in DTC workflows, and propose a set of criteria to evaluate a simulation tool’s applicability to DTC.

Authors: Li B., Nielsen R.O., Johansen K.W., Teizer J. , Larsen P.G., Schultz C.
University: Department of Electrical and Computer Engineering – Department of Civil and Architecture Engineering, Aarhus University, Denmark

Civil engineering has only recently started the digitalisation journey by standardising around Building Information Models (BIMs). In the process of construction a dimension of time is added in what is called 4D BIM and this can serve as the basis for a digital twin. It is predicted that such a digital twin can enhance the overall overview of status of the construction of a new building by means of different types of sensors, and interpreting these in relation to a BIM. In the construction phase there are rules and regulations targeting the safety of the different kinds of construction workers at the construction site. In this paper we provide a vision of how digital twins can assist with spotting potential violations of the constraints stated by the rules and regulations, and empirically evaluate a proof-of-concept software tool on a large scale, real-world 4D BIM.