CN-121982498-A - Indoor construction progress visualization mapping method integrating semantic recognition and space positioning

CN121982498ACN 121982498 ACN121982498 ACN 121982498ACN-121982498-A

Abstract

The invention discloses an indoor construction progress visualization mapping method integrating semantic recognition and space positioning, which comprises the following steps of adopting a panoramic camera to collect multi-frame image data of all components of an indoor construction scene, outputting semantic segmentation results corresponding to all the components in each frame of image, synchronously obtaining camera pose data and scene sparse point cloud data which are in one-to-one correspondence with each frame of image timestamp, constructing a semantic sparse point cloud map containing component category information, performing similarity transformation based on scale factors and camera pose data, mapping the semantic sparse point cloud map from an SLAM world coordinate system to a pixel coordinate system of a building plan, removing repeated projection of the same component caused by continuous collection of multi-frame images, carrying out weighted summation to obtain an indoor construction progress value, and carrying out visual display on the progress value and the component mapping result on the building plan. The invention has the advantages of solving the problems of automation, accuracy and visual monitoring of the indoor construction progress.

Inventors

GAO ZECHENG
ZHANG LIWEN
Gao Zhangchao

Assignees

上海市隧道工程轨道交通设计研究院

Dates

Publication Date: 20260505
Application Date: 20260109

Claims (8)

1. A visual mapping method for indoor construction progress integrating semantic recognition and space positioning is characterized by comprising the following steps: S1, acquiring multi-frame image data of each component of an indoor construction scene by adopting a panoramic camera; S2, inputting the multi-frame image data acquired in the step S1 into an improved semantic segmentation model, wherein the improved semantic segmentation model carries out smooth confidence level attenuation on an overlapping detection frame by replacing NMS in an example segmentation branch with Soft-NM, and outputs a semantic segmentation result corresponding to each component in each frame of image; S3, based on a visual SLAM frame, tracking the motion trail of the camera in the image acquisition process in the step S1 in real time, and synchronously acquiring camera pose data and scene sparse point cloud data which are in one-to-one correspondence with each frame of image time stamp; s4, aligning the semantic segmentation result output by the step S2 with the corresponding camera pose data acquired by the step S3 through a time stamp, calling a camera internal reference matrix K and a pinhole projection model, and back-projecting the semantic segmentation result of each frame of image into the scene sparse point cloud data of the step S3 to construct a semantic sparse point cloud map containing component category information; S5, setting a calibration plate with a known size L real in a construction scene, solving the real pose of the camera by a PnP algorithm in combination with a camera internal reference matrix K, calculating a scale factor S scale by utilizing the ratio of the known size L real of the calibration plate to the distance L norm of a corresponding angular point of the calibration plate in a visual SLAM normalization space, performing similar transformation on the basis of the camera pose data acquired in the scale factor S scale and the step S3, mapping a semantic sparse point cloud map constructed in the step S4 from a SLAM world coordinate system to a pixel coordinate system of a building plan, and obtaining a pixel coordinate point (u map ,v map ) containing a component type label as a semantic feature point; S6, processing the semantic feature points obtained in the step S5 by adopting a density clustering algorithm, aggregating high-density semantic feature point areas through presetting a clustering radius and minimum points, removing isolated noise points to obtain clustering clusters corresponding to each component, calculating the minimum circumscribed rectangle of each clustering cluster, and describing the geometric outer boundary of the corresponding component approximately by the minimum circumscribed rectangle; S7, presetting the respective maximum possible instance number N i of each component in a reference scene, counting the detection instance number N i of each type of component in the effective mapping area in the step S6, presetting a contribution degree weight omega i of a construction stage according to the importance of the component in a construction flow, carrying out weighted summation to obtain a progress value P of indoor construction, and carrying out visual display on the progress value P and a component mapping result on a building plan.
2. The visual mapping method for indoor construction progress integrating semantic recognition and space positioning according to claim 1 is characterized in that in the step S1, the panoramic camera is fixed on a security helmet of an inspection personnel, the component comprises at least one of a wall body, a structural column, a constructional column, a masonry material, a cement material and a wooden template, and the panoramic camera synchronously records time stamps of each frame of image.
3. The visual mapping method of indoor construction progress integrating semantic recognition and space positioning according to claim 2, wherein in step S2, the improved semantic segmentation model is an improved Panoptic-FPN model; The confidence decay formula of Soft-NMS is: ; Wherein s i is the confidence coefficient of the ith detection frame, ioU (i, j) is the overlapping rate of the ith detection frame and the highest confidence detection frame j, N t is an overlapping rate threshold value of an adaptive indoor shielding scene, and sigma is a coefficient for controlling the attenuation smoothness degree; The semantic segmentation result includes a component category label and a pixel region mask.
4. The visual mapping method for indoor construction progress integrating semantic recognition and space positioning according to claim 3 is characterized in that in the step S3, the visual SLAM frame is OpenVSLAM frames, the camera pose data comprise a rotation matrix R and a translation vector t, and the sparse point cloud data are coordinate sets P ω =（X ω ,Y ω ,Z ω of three-dimensional space feature points of a scene.
5. The visual mapping method of indoor construction progress integrating semantic recognition and space positioning as claimed in claim 4, wherein in step S4, a camera internal parameter matrix K is defined as: ; Wherein, f x 、f y is the focal length of the camera in the x-axis direction and the y-axis direction respectively, and c x 、c y is the x-axis direction coordinate and the y-axis direction coordinate of the imaging center of the camera respectively; The back projection formula is: ; ; ; Where P c =（X c ,Y c ,Z c ) is the coordinates of the spatial point in the camera coordinate system, and (u, v) is the coordinates in the pixel coordinate system.
6. The visual mapping method of indoor construction progress integrating semantic recognition and space positioning according to claim 5, wherein in step S5, the pose solving formula of PnP algorithm is: ; wherein s is a scale coefficient of a pixel coordinate system of the building plan and a SLAM world coordinate system; calculating a scale factor S scale : ; The similarity transformation formula is: ; Where t map is the translational compensation vector of the pixel coordinate system of the building plan and the SLAM world coordinate system.
7. The visual mapping method of indoor construction progress integrating semantic recognition and space positioning according to claim 6, wherein in step S6, the density clustering algorithm is a DBSCAN algorithm; IoU the calculation formula is: ; wherein, MBR i 、MBR j is the minimum circumscribed rectangle of any two components, and Area (&) is the rectangular Area; IoU has a threshold of 0.5.
8. The visual mapping method of indoor construction progress integrating semantic recognition and space positioning according to claim 7, wherein in step S7, in the contribution degree weight omega i of the construction stage, the weight of a forming member is higher than that of a material member, the forming member comprises a wall body, a structural column and a constructional column, and the material member comprises a masonry material, a cement material and a wooden template; and obtaining the progress value P of indoor construction by weighting and summing according to the following formula: ; In the formula, The progress value is ensured to be within the [0,1] interval.

Description

Indoor construction progress visualization mapping method integrating semantic recognition and space positioning Technical Field The invention relates to the technical field of digital management, in particular to an indoor construction progress visualization mapping method integrating semantic recognition and space positioning. Background At present, the related prior art for monitoring the indoor construction progress mainly comprises the following three types: ① The traditional manual inspection technology is that the inspection personnel observe and record the completion condition of the components on site, and the progress comparison is carried out by combining paper drawings or BIM models, so that the method is a main stream mode for monitoring the progress of the indoor construction at present; ② A single computer vision technology, which is to identify the component category in the construction scene based on a deep learning semantic segmentation model (such as Panoptic-FPN and Mask R-CNN), but only output semantic information in a two-dimensional image, lack of space positioning capability and cannot be associated with a building plan; ③ A single space positioning technology, which is to acquire a camera track and sparse point cloud of a construction scene based on visual SLAM (such as OpenVSLAM) to realize space positioning and map building, but cannot distinguish semantic categories of components, so that the evaluation of the progress is difficult to directly support; ④ Outdoor adaptation comprehensive technologies such as unmanned aerial vehicle photogrammetry and BIM+ three-dimensional laser scanning can realize outdoor construction progress monitoring, but are limited by environments with narrow indoor space and serious shielding, cannot flexibly operate, and have insufficient adaptability. The prior art has the following defects that can not be overcome, so that the indoor construction progress monitoring is difficult to meet the requirements of refinement and high-efficiency management: ① The automatic degree is low, the traditional manual inspection relies on manual recording and comparison, the data is lagged, the efficiency is low, the progress evaluation accuracy depends on personnel experience, and the subjectivity is strong; ② The semantic and spatial information are disjoint, namely the single semantic segmentation technology lacks spatial positioning capability, the single spatial positioning technology lacks semantic information, and the association mapping of component category-spatial position-plane drawing cannot be realized; ③ The indoor environment adaptability is poor, the outdoor adaptation technology cannot cope with the scenes with narrow indoor space, serious shielding and changeable illumination, and the component identification missing detection and the false detection rate are high; ④ The mapping redundancy and the accuracy are insufficient, namely repeated projection of semantic feature points is easily caused by multi-frame image acquisition, the prior art lacks an effective de-duplication scheme, and the evaluation accuracy of interference progress is poor; ⑤ The progress evaluation standardization is insufficient, namely, the prior art does not establish a standardized progress evaluation model based on the semantic features of the components, and objective and accurate progress quantitative calculation cannot be realized. Disclosure of Invention According to the defects of the prior art, the invention provides the visual mapping method for the indoor construction progress, which is used for integrating semantic recognition and space positioning, and solves the problems of automatic, accurate and visual monitoring of the indoor construction progress by integrating computer vision and space positioning technology, thereby providing technical support for engineering construction collaborative management and construction period control. The invention is realized by the following technical scheme: A visual mapping method for indoor construction progress integrating semantic recognition and space positioning comprises the following steps: S1, acquiring multi-frame image data of each component of an indoor construction scene by adopting a panoramic camera; S2, inputting the multi-frame image data acquired in the step S1 into an improved semantic segmentation model, wherein the improved semantic segmentation model carries out smooth confidence level attenuation on an overlapping detection frame by replacing NMS in an example segmentation branch with Soft-NM, and outputs a semantic segmentation result corresponding to each component in each frame of image; S3, based on a visual SLAM frame, tracking the motion trail of the camera in the image acquisition process in the step S1 in real time, and synchronously acquiring camera pose data and scene sparse point cloud data which are in one-to-one correspondence with each frame of image time stamp; s4, aligning the semantic segme