Chapter 24
An Integrated AR-FEM-AI System for Real-time Structural Simulations and X-ray Visualisation of Heritage and Ageing Buildings
Iasef Md Rian
Department of Architecture, Xian Jiaotong-Liverpool University, Suzhou, China
Abstract
Heritage and ageing structures are increasingly vulnerable to deterioration, environmental loads, and long-term material fatigue. Conventional structural monitoring techniques, such as embedded sensing systems or invasive sampling, often face practical, financial, or ethical constraints, particularly in conservation contexts. In response, this research proposes an integrated, non-contact framework that combines Augmented Reality (AR), Finite Element Modelling (FEM), and Artificial Intelligence (AI) to enable early-stage structural assessment and immersive visualisation. The framework is currently under development and is being applied to representative case studies of Chinese heritage architecture, including timber pagodas and masonry towers. Within this system, FEM provides simulated structural responses under various loading scenarios. AI models are trained on these outputs to deliver rapid, predictive analysis, and AR interfaces project these insights onto physical structures through intuitive, layered visualizations. This approach aims to support conservation workflows by facilitating preliminary diagnostics without physical intervention, while also enhancing educational and public engagement with built heritage. Although still in progress, the framework is conceived to be scalable and adaptable across diverse structural typologies and regional contexts, offering a novel pathway toward sustainable, data-informed heritage management.
Keywords: Artificial Intelligence, Augmented Reality, FEM, Predictive Simulation, Structural Health Monitoring, Heritage, Ageing and Existing Structures, Digital Twin
Introduction and Background
Heritage structures, from Gothic cathedrals in Europe to timber pagodas in China, embody centuries of cultural and technological achievement. Yet they face accelerating risks from ageing, climate change, urbanisation, and inadequate maintenance1. Modern urban buildings, though constructed with advanced materials, also develop vulnerabilities over time, highlighting the need for advanced monitoring strategies to ensure resilience and safety2. Traditional diagnostic methods such as strain gauges, embedded sensors, and destructive sampling are often invasive, expensive, and limited in scope. For heritage sites protected under conservation laws, such approaches are frequently unacceptable, while large-scale or retrofitted buildings demand real-time solutions that surpass the capabilities of manual inspection3.
Recent advances in digital technology present opportunities to transform structural monitoring through the integration of computational modelling and immersive visualisation (Figure 1). Augmented Reality (AR) offers interactive ways to engage with heritage, but current applications tend to emphasise appearance rather than performance4. The Finite Element Method (FEM) remains the benchmark for high-fidelity simulation of stresses, deformations, and dynamic responses, yet its computational demands hinder real-time use5. Artificial Intelligence (AI), particularly machine learning, provides a bridge by learning from FEM datasets and sensor inputs to generate predictive surrogates capable of rapid, scalable analysis6.
Figure 1: Previous studies and simulations of Chinese heritage structures.
Heritage buildings present particular challenges due to their material variability, aged timber, stone, or rammed earth often resist standardised characterisation, and conservation restrictions that limit invasive intervention7. Environmental stressors such as seismic activity, moisture infiltration, and wind further accelerate deterioration. Comparable problems are evident in ageing modern structures, where concrete fatigue, steel corrosion, and incompatibility between old and new systems complicate maintenance8. These factors underscore the urgent need for non-invasive yet accurate monitoring frameworks.
This research proposes a novel integration of AR, FEM, and AI into a single system for real-time, predictive structural analysis and immersive X-ray visualisation. The framework is being designed initially for testing on Chinese heritage buildings such as the Xuanmiao Guan Temple (Suzhou), the Yingxian Wooden Pagoda (Yingxian), and the Tiger Hill Pagoda (Suzhou) (Figure 2), chosen for their material diversity, architectural form, structural typology, and cultural significance (Figure 3). While rooted in Chinese case studies, the system is inherently adaptable to diverse global typologies of heritage and ageing buildings, offering a contribution to conservation, education, and structural resilience9.
Figure 2: Selected Chinese heritage structures for pilot study and implementation of the AR-FEM-AI system.
Figure 3: Conceptual workflow of the AI-FEM-AR system.
Aims and Objectives
This research aims to develop and validate a unified AR-FEM-AI system to support preliminary structural health scanning and immersive visualisation of historical buildings (Figure 3). Intended as a first-response tool, the system offers hands-off, data-driven insights that help identify potential structural issues early, guiding conservation professionals toward areas requiring detailed analysis. It also serves as an educational and engagement platform for educators and the public. Key objectives include:
a. Developing an AR platform that overlays real-time indicators, such as stress, deformation, and risk zones, onto heritage structures for rapid visual assessment.
b. Conducting FEM simulations on representative Chinese heritage buildings to generate foundational datasets for AI training.
c. Designing AI models that learn from FEM outputs and sensor inputs to provide fast, adaptive predictions suitable for early-stage evaluations.
d. Validating the integrated system through pilot applications at selected heritage sites, assessing its effectiveness as a diagnostic tool and its broader impact on conservation workflows, education, and public engagement.
Methodology
The methodology follows an iterative, four-step workflow currently in progressstarting from digital capture and progressing through simulation, AI training, and AR-based visualisation. Each phase is interconnected, supporting the development of a responsive, data-informed tool for early-stage structural health assessment (Figure 4).
Step 1: Digital Twin Construction
Tools: High-resolution 3D scanning, photogrammetry, archival research
The process begins with the creation of accurate digital twins of heritage structures. These models are being developed using high-resolution scanning technologies that capture both geometric form and surface texture. Archival documents and historical references are used to estimate hidden structural components and material properties. The resulting models serve not only as visual replicas but also as data-rich foundations for structural analysis.
Figure 4: Methodological workflow for developing the AI-FEM-AR system.
Step 2: Structural Simulation via FEM
Tools: Finite Element Modelling software (e.g., ANSYS, Abaqus, Karamba3D)
These digital twins are currently being analysed through Finite Element Modelling (FEM). Simulations emphasise different structural behaviours depending on the material and typology, such as timber anisotropy in wooden pagodas, nonlinear cracking in masonry, or corrosion in reinforced concrete. FEM outputs include predicted stress distributions, displacements, modal frequencies, and failure zones. However, their high computational demands limit their use in real-time applications.
Step 3: AI Surrogate Model Training
Tools: Machine learning libraries (e.g., TensorFlow, PyTorch), neural network models
To overcome FEMs computational constraints, AI surrogate models are being developed. Convolutional and recurrent neural networks are trained on FEM-generated datasets to learn how input parameters (e.g., load conditions, boundary constraints) relate to structural responses. Once adequately trained, these AI models are expected to deliver near-instantaneous predictions, enabling rapid assessments in the field.
Step 4: Immersive AR Visualisation
Tools: AR hardware (e.g., Microsoft HoloLens, AR-enabled tablets)
The final step involves integrating the trained AI predictions into an AR platform, which is under development. This interface will overlay real-time structural insights, such as stress paths and deformation zones, onto physical buildings. Engineers, conservators, and even the public will be able to interact directly with the structure, gaining insights through visual overlays without physical contact.
Expected and Preliminary Results
The integrated system is designed as an early-stage diagnostic tool for structural health monitoringoffering conservation teams a first-look assessment that can inform expert decision-making and guide more advanced investigations. Rather than replacing high-fidelity FEM analysis, it serves as a rapid, non-invasive scanning approach to flag potential structural issues and prioritise areas for deeper study.
The prototype allows engineers to navigate heritage sites while viewing projected overlays of stress patterns and deformation directly on structural elements. For conservators, this transforms preliminary inspections into interactive, data-informed processes. The system also enhances public engagement by enabling tourists and students to explore the hidden structural dynamics of historic architecture.
The Yingxian Wooden Pagodas roof frame is currently being used as the first pilot case (Figure 5). The team is preparing 3D scanning to generate a refined FEM model, while experimenting with various Dougong bracket types. Roof structures across all tiers are being processed for machine learning training. These efforts will lay the foundation for the next development phase of the AR-FEM-AI platform.
Discussion: Potential, Limitations, and Future Work
This research aimed to develop a unified AR-FEM-AI system for early-stage structural health monitoring and visualisation of heritage buildings. By combining the analytical precision of FEM, the responsiveness of AI, and the accessibility of AR, the system aims to offer a hands-off alternative to traditional methods, supporting rapid, first-pass assessments to guide expert analysis.
Initial applications to Chinese timber and reinforced-concrete structures are showing promise. Ongoing and future work focuses on expanding compatibility to include stone masonry, steel, and hybrid typologies to improve the generalizability of AI models. Integration with IoT-based sensors is being explored to enable real-time updates, while a hybrid feedback loop combining simulated and measured data is expected to enhance prediction accuracy. Planned VR extensions will support remote collaboration and broaden applications in conservation, education, and disaster risk management.
However, certain limitations are already evident. AI performance depends on the quality of FEM training data, which is often constrained by undocumented materials and limited access to physical testing. AR deployment may face spatial misalignment, lag, or environmental interference in field conditions. Current development efforts focus on adaptive learning for incomplete data, transfer learning across similar structures, and lightweight AR interfaces.
Figure 5: Top Different Douging typologies of Yingxian Wooden Pagoda; Bottom FEM structural simulations of all dissimilar Dougong bracket types.
Beyond technical goals, the platform aims to foster public engagement by revealing hidden structural narratives. As development progresses, it has the potential to evolve into a scalable tool for sustainable heritage conservation, bridging technology and cultural preservation.
Acknowledgements
This work is being carried out using the facilities of Xian Jiaotong-Liverpool University without external funding. The team welcomes opportunities for future research grants in this area.
References
Bitik, Doga. 2024. Climate- Responsive HBIM for Sustainable Preservation: A Case Study of Villa Ottolini Tovaglieri - Environmental Impact and Preservation Strategies. Polimi.it, June. https://hdl.handle.net/10589/223026.
Catbas, Fikret Necati, Furkan Luleci, Mahta Zakaria, Ulas Bagci, Joseph J. LaViola Jr, Carolina Cruz-Neira, and Dirk Reiners. 2022. "Extended reality (XR) for condition assessment of civil engineering structures: A literature review." Sensors 22 (23): 9560. https://doi.org/10.3390/ s22239560.
De Fino, Mariella, Rosella Alessia Galantucci, and Fabio Fatiguso. 2023. "Condition assessment of heritage buildings via photogrammetry: a scoping review from the perspective of decision makers." Heritage 6 (11): 7031-7066. https://doi.org/10.3390/heritage6110367.
Laohaviraphap, Neeraparng, and Tanut Waroonkun. 2024. "Integrating Artificial Intelligence and the Internet of Things in Cultural Heritage Preservation: A Systematic Review of Risk Management and Environmental Monitoring Strategies." Buildings 14 (12): 3979. https://doi.org/10.3390/ buildings14123979.
Lignola, Gian Piero, Nicola Buratti, Serena Cattari, Fulvio Parisi, Filippo Ubertini, Sara Alfano, Laura Ierimonti, Andrea Meoni, Daniele Sivori, and Giorgio Virgulto. 2025. "Validated and Optimized Strategies for Preserving Historical Heritage Towards Natural and Anthropic Risks: Insights from the DETECT-AGING Project." Buildings 15 (5): 693. https://doi.org/10.3390/ buildings15050693.
Plevris, Vagelis, and George Papazafeiropoulos. 2024. "AI in structural health monitoring for infrastructure maintenance and safety." Infrastructures 9 (12): 225. https://doi.org/10.3390/ infrastructures9120225.
Yu, Yingwen, Abeer Abu Raed, Yuyang Peng, Uta Pottgiesser, Edward Verbree, and Peter van Oosterom. 2024. "How digital technologies have been applied for architectural heritage risk management: a systemic literature review from 2014 to 2024." npj Heritage Science 13 (1): 45. https://doi.org/10.1038/s40494-025-01558-5.
Rossi, Michela, and Dionysios Bournas. 2023. "Structural health monitoring and management of cultural heritage structures: a state-of-the-art review." Applied Sciences 13 (11): 6450. https:// doi.org/10.3390/app13116450.
Serbouti, Imane, Jrme Chenal, Sad Abdesslam Tazi, Ahmad Baik, and Mustapha Hakdaoui. 2025. "Digital transformation in African heritage preservation: A digital twin framework for a sustainable Bab Al-Mansour in Meknes City, Morocco." Smart Cities 8 (1): 29. https:// doi.org/10.3390/smartcities8010029.
Wang, Jingyi, and Safial Aqbar Zakaria. 2025. "Design Application and Evolution of 3D Visualization Technology in Architectural Heritage Conservation: A CiteSpace-Based Knowledge Mapping and Systematic Review (20052024)." Buildings 15 (11): 1854. https://doi.org/10.3390/ buildings15111854.