A Review of Techniques for 3D Reconstruction of Indoor Environments

Surface Defect-Extended BIM Generation Leveraging UAV Images and Deep Learning

Defect inspection of existing buildings is receiving increasing attention for digitalization transfer in the construction industry. The development of drone technology and artificial intelligence has provided powerful tools for defect inspection of buildings. However, integrating defect inspection information detected from UAV images into semantically rich building information modeling (BIM) is still challenging work due to the low defect detection accuracy and the coordinate difference between UAV images and BIM models. In this paper, a deep learning-based method coupled with transfer learning is used to detect defects accurately; and a texture mapping-based defect parameter extraction method is proposed to achieve the mapping from the image U-V coordinate system to the BIM project coordinate system. The defects are projected onto the surface of the BIM model to enrich a surface defect-extended BIM (SDE-BIM). The proposed method was validated in a defect information modeling experiment involving the No. 36 teaching building of Nantong University. The results demonstrate that the methods are widely applicable to various building inspection tasks.

3D Visual Reconstruction as Prior Information for First Responder Localization and Visualization

In professional use cases like police or fire brigade missions, coordinated and systematic force management is crucial for achieving operational success during intervention by the emergency personnel. A real-time situation picture enhances the coordination of the team. This situation picture includes not only an overview of the environment but also the positions, i.e., localization, of the emergency forces. The overview of the environment can be obtained either from known situation pictures like floorplans or by scanning the environment with the aid of visual sensors. The self-localization problem can be solved outdoors using the Global Navigation Satellite System (GNSS), but it is not fully solved indoors, where the GNSS signal might not be received or might be degraded. In this paper, we propose a novel combination of an inertial localization technique based on simultaneous localization and mapping (SLAM) with 3D building scans, which are used as prior information, for geo-referencing the positions, obtaining a situation picture, and finally visualizing the results with an appropriate visualization tool. We developed a new method for converting point clouds into a hexagonal prism map specifically designed for our SLAM algorithm. With this combination, we could keep the equipment for first responders as lightweight as required. We showed that the positioning led to an average accuracy of less than 1m indoors, and the final visualization including the building layout obtained by the 3D building reconstruction will be advantageous for coordinating first responder operations.

Generating Topologically Consistent BIM Models of Utility Tunnels from Point Clouds

The development and utilization of urban underground space is an important way to solve the "great urban disease". As one of the most important types of urban underground foundations, utility tunnels have become increasingly popular in municipal construction. The investigation of utility tunnels is a general task and three-dimensional laser scanning technology has played a significant role in surveying and data acquisition. However, three-dimensional laser scanning technology suffers from noise and occlusion in narrow congested utility tunnel spaces, and the acquired point clouds are imperfect; hence, errors and redundancies are introduced in the extracted geometric elements. The topology of reconstructed BIM objects cannot be ensured. Therefore, in this study, a hierarchical segmentation method for point clouds and a topology reconstruction method for building information model (BIM) objects in utility tunnels are proposed. The point cloud is segmented into facades, planes, and pipelines hierarchically. An improved mean-shift algorithm is proposed to extract wall line features and a local symmetry-based medial axis extraction algorithm is proposed to extract pipelines from point clouds. A topology reconstruction method that searches for the neighbor information of wall and pipeline centerlines and establishes collinear, perpendicular, and intersecting situations is used to reconstruct a topologically consistent 3D model of a utility tunnel. An experiment on the Guangzhou's Nansha District dataset successfully reconstructed 24 BIM wall objects and 12 pipelines within the utility tunnel, verifying the efficiency of the method.

Progressive Model-Driven Approach for 3D Modeling of Indoor Spaces

This paper focuses on the 3D modeling of the interior spaces of buildings. Three-dimensional point clouds from laser scanners can be considered the most widely used data for 3D indoor modeling. Therefore, the walls, ceiling and floor are extracted as the main structural fabric and reconstructed. In this paper, a method is presented to tackle the problems related to the data including obstruction, clutter and noise. This method reconstructs indoor space in a model-driven approach using watertight predefined models. Employing the two-step implementation of this process, the algorithm is able to model non-rectangular spaces with an even number of sides. Afterwards, an "improvement" process increases the level of details by modeling the intrusion and protrusion of the model. The 3D model is formed by extrusion from 2D to 3D. The proposed model-driven algorithm is evaluated with four benchmark real-world datasets. The efficacy of the proposed method is proved by the range of [77%, 95%], [85%, 97%] and [1.7 cm, 2.4 cm] values of completeness, correctness and geometric accuracy, respectively.

Robo-vision! 3D mesh generation of a scene for a robot for planar and non-planar complex objects

This paper presents a novel architecture to generate a world model in terms of mesh from a continuous image stream with depth information extracted from a robot's ego-vision camera. We propose two algorithms for planar and non-planar mesh generation. A Cartesian grid-based mesh fitting algorithm is proposed for mesh generation of planar objects. For mesh generation of non-planar objects, we propose a Self Organization Map based algorithm. The proposed algorithm better approaches the boundary and overall shape of the objects compared to State-Of-the-Art (SOA). Extensive experiments done on three public datasets show that our method surpasses SOA both qualitatively and quantitatively.

Modeling and Optimization of the Para-Xylene Continuous Suspension Crystallization Separation Process via a Morphology Technique and a Multi-Dimensional Population Balance Equation

In this study, we carried out a para-xylene crystallization experiment at constant temperature and concentration levels. Throughout the process, the kinetics of nucleation, growth, breakage, and aggregation of para-xylene particles were measured and built using a morphological approach. An additional a three-stage continuous suspension crystallization separation experiment was carried out, the process for which was simulated using the population balance model based on correlated kinetic equations. The population balance equation was solved using an extended moment of classes algorithm, and the solving process was implemented in MATLAB. In this case, the predicted particle size distribution of the products matched well with the experiment. In order to provide references for the optimization of the industrial para-xylene crystallization process, a three-stage suspension crystallization separation experiment was designed and conducted, in which each crystallizer had a distinct operating temperature and mean residence time. The effects of operating parameters on the final product were investigated further. The proposed models and algorithms can also be applied in other cases and provide an alternative approach for optimizing continuous crystallization processes.

Robot Path Planning Method Based on Indoor Spacetime Grid Model

In the context of digital twins, smart city construction and artificial intelligence technology are developing rapidly, and more and more mobile robots are performing tasks in complex and time-varying indoor environments, making, at present, the unification of modeling, dynamic expression, visualization of operation, and wide application between robots and indoor environments a pressing problem to be solved. This paper presents an in-depth study on this issue and summarizes three major types of methods: geometric modeling, topological modeling, and raster modeling, and points out the advantages and disadvantages of these three types of methods. Therefore, in view of the current pain points of robots and complex time-varying indoor environments, this paper proposes an indoor spacetime grid model based on the three-dimensional division framework of the Earth space and innovatively integrates time division on the basis of space division. On the basis of the model, a dynamic path planning algorithm for the robot in the complex time-varying indoor environment is designed, that is, the Spacetime-A* algorithm (STA* for short). Finally, the indoor spacetime grid modeling experiment is carried out with real data, which verifies the feasibility and correctness of the spacetime relationship calculation algorithm encoded by the indoor spacetime grid model. Then, experiments are carried out on the multi-group path planning algorithms of the robot under the spacetime grid, and the feasibility of the STA* algorithm under the indoor spacetime grid and the superiority of the spacetime grid are verified.

Topologically Consistent Reconstruction for Complex Indoor Structures from Point Clouds

Indoor structures are composed of ceilings, walls and floors that need to be modeled for a variety of applications. This paper proposes an approach to reconstructing models of indoor structures in complex environments. First, semantic pre-processing, including segmentation and occlusion construction, is applied to segment the input point clouds to generate semantic patches of structural primitives with uniform density. Then, a primitives extraction method with detected boundary is introduced to approximate both the mathematical surface and the boundary of the patches. Finally, a constraint-based model reconstruction is applied to achieve the final topologically consistent structural model. Under this framework, both the geometric and structural constraints are considered in a holistic manner to assure topologic regularity. Experiments were carried out with both synthetic and real-world datasets. The accuracy of the proposed method achieved an overall reconstruction quality of approximately 4.60 cm of root mean square error (RMSE) and 94.10% Intersection over Union (IoU) of the input point cloud. The development can be applied for structural reconstruction of various complex indoor environments.

Survey Solutions for 3D Acquisition and Representation of Artificial and Natural Caves

A three-dimensional survey of natural caves is often a difficult task due to the roughness of the investigated area and the problems of accessibility. Traditional adopted techniques allow a simplified acquisition of the topography of caves characterized by an oversimplification of the geometry. Nowadays, the advent of LiDAR and Structure from Motion applications eased three-dimensional surveys in different environments. In this paper, we present a comparison between other three-dimensional survey systems, namely a Terrestrial Laser Scanner, a SLAM-based portable instrument, and a commercial photo camera, to test their possible deployment in natural caves survey. We presented a comparative test carried out in a tunnel stretch to calibrate the instrumentation on a benchmark site. The choice of the site is motivated by its regular geometry and easy accessibility. According to the result obtained in the calibration site, we presented a methodology, based on the Structure from Motion approach that resulted in the best compromise among accuracy, feasibility, and cost-effectiveness, that could be adopted for the three-dimensional survey of complex natural caves using a sequence of images and the structure from motion algorithm. The methods consider two different approaches to obtain a low resolution complete three-dimensional model of the cave and ultra-detailed models of most peculiar cave morphological elements. The proposed system was tested in the Gazzano Cave (Piemonte region, Northwestern Italy). The obtained result is a three-dimensional model of the cave at low resolution due to the site’s extension and the remarkable amount of data. Additionally, a peculiar speleothem, i.e., a stalagmite, in the cave was surveyed at high resolution to test the proposed high-resolution approach on a single object. The benchmark and the cave trials allowed a better definition of the instrumentation choice for underground surveys regarding accuracy and feasibility.

Co-Training for Deep Object Detection: Comparing Single-Modal and Multi-Modal Approaches

Top-performing computer vision models are powered by convolutional neural networks (CNNs). Training an accurate CNN highly depends on both the raw sensor data and their associated ground truth (GT). Collecting such GT is usually done through human labeling, which is time-consuming and does not scale as we wish. This data-labeling bottleneck may be intensified due to domain shifts among image sensors, which could force per-sensor data labeling. In this paper, we focus on the use of co-training, a semi-supervised learning (SSL) method, for obtaining self-labeled object bounding boxes (BBs), i.e., the GT to train deep object detectors. In particular, we assess the goodness of multi-modal co-training by relying on two different views of an image, namely, appearance (RGB) and estimated depth (D). Moreover, we compare appearance-based single-modal co-training with multi-modal. Our results suggest that in a standard SSL setting (no domain shift, a few human-labeled data) and under virtual-to-real domain shift (many virtual-world labeled data, no human-labeled data) multi-modal co-training outperforms single-modal. In the latter case, by performing GAN-based domain translation both co-training modalities are on par, at least when using an off-the-shelf depth estimation model not specifically trained on the translated images.

A Comparison of Low-Cost Sensor Systems in Automatic Cloud-Based Indoor 3D Modeling

The automated 3D modeling of indoor spaces is a rapidly advancing field, in which recent developments have made the modeling process more accessible to consumers by lowering the cost of instruments and offering a highly automated service for 3D model creation. We compared the performance of three low-cost sensor systems; one RGB-D camera, one low-end terrestrial laser scanner (TLS), and one panoramic camera, using a cloud-based processing service to automatically create mesh models and point clouds, evaluating the accuracy of the results against a reference point cloud from a higher-end TLS. While adequately accurate results could be obtained with all three sensor systems, the TLS performed the best both in terms of reconstructing the overall room geometry and smaller details, with the panoramic camera clearly trailing the other systems and the RGB-D offering a middle ground in terms of both cost and quality. The results demonstrate the attractiveness of fully automatic cloud-based indoor 3D modeling for low-cost sensor systems, with the latter providing better model accuracy and completeness, and with all systems offering a rapid rate of data acquisition through an easy-to-use interface.