CN-122024170-A - Construction enclosure structural change detection and multi-mode auxiliary treatment method and system

CN122024170ACN 122024170 ACN122024170 ACN 122024170ACN-122024170-A

Abstract

The invention discloses a method and a system for detecting structural change of a construction fence and assisting in multi-mode treatment, which belong to the field of safety management of constructional engineering and comprise the following steps of collecting a current image of a fence area, and registering and aligning the current image with a base reference image; respectively extracting feature images of the two registered images, generating a depth image based on the current feature image, fusing the reference feature image, the current feature image and the depth image for change detection, outputting a boundary frame and a type label of a change area, combining the depth image and the camera internal parameter to estimate the physical size of the change area, constructing a structuring result based on the boundary frame, the type label, the physical size and the image, inputting the structuring result into a multi-mode large language model, outputting semantic interpretation and maintenance advice, and generating a maintenance work order according to the structuring result. The invention greatly improves the intelligent level and response efficiency of construction safety management.

Inventors

ZHANG XIAOMING
YANG KANG
YAN JINGQI
ZHAO HUI
JIANG ZHENGLONG
ZHONG SHENG

Assignees

上海同陆云交通科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260226

Claims (10)

1. The method for detecting the structural change of the construction enclosure and assisting in treatment in a multi-mode manner is characterized by comprising the following steps of: acquiring an image of a construction enclosure area, acquiring a current image, and processing the current image and a base reference image by adopting an image registration method to obtain a registered current image; Processing by adopting a feature extraction model based on the base reference image and the registered current image to obtain a reference image feature image and a current image feature image; Based on the current image feature map, processing by adopting a depth estimation model to obtain a current image depth map; Processing the reference image feature map, the current image feature map and the current image depth map by adopting a change detection model to obtain a boundary frame coordinate and a change type label of a surrounding change area; Calculating the current image depth map, the camera internal parameters and the boundary frame coordinates to obtain the physical size of the change area; Constructing a structured detection result based on the boundary frame coordinates, the change type label, the physical size and the current image; based on the structural detection result, processing by adopting a multi-mode large language model, and outputting semantic interpretation and maintenance suggestions; and generating a maintenance work order based on the structural detection result and the semantic interpretation and maintenance suggestion.
2. The method for detecting structural changes of construction fence and for multi-modal assisted treatment according to claim 1, wherein the processing of the current image and the baseline reference image using an image registration method comprises: Based on the base reference image and the current image, processing by adopting a SIFT feature point detection algorithm, and extracting corresponding feature point pairs; based on the corresponding characteristic point pairs, adopting a RANSAC robust fitting method to process, and estimating a two-dimensional homography matrix; and performing geometric transformation processing based on the two-dimensional homography matrix and the current image to obtain the registered current image.
3. The method for detecting structural changes of construction fence and assisting treatment in multiple modes according to claim 1, wherein the process of processing the reference image feature map, the current image feature map and the current image depth map by using a change detection model comprises: A change region proposal stage, calculating feature differences based on the reference image feature map and the current image feature map, and combining high-frequency enhancement and a surrounding region of interest mask for processing to generate a change region candidate boundary frame; And in the change category judging stage, based on the current image feature map, the current image depth map and the change region candidate boundary box, feature and depth fusion is carried out, and the change type label is obtained by searching a pre-constructed feature library for processing.
4. A method of construction enclosure structural change detection and multimodal assisted treatment according to claim 3, wherein the process of calculating the characteristic differences comprises: respectively carrying out L2 normalization processing on the feature vector in the reference image feature map and the corresponding feature vector in the current image feature map to obtain a normalized feature vector; calculating cosine distance and L2 differential norm based on the normalized feature vector; And respectively carrying out normalization processing on the cosine distance graph and the L2 differential norm graph, and then carrying out linear fusion to obtain a basic change significance graph.
5. A method of construction enclosure structural change detection and multi-modal auxiliary treatment according to claim 3, wherein the construction process of the feature library comprises: Based on the image of the history enclosure change sample, executing the steps of feature extraction, depth estimation and change region proposal to obtain candidate regions; based on the current image feature map and the current image depth map of the candidate region, performing feature and depth fusion processing to obtain fusion features; flattening the fusion characteristics and performing dimension reduction processing based on principal component analysis to obtain dimension reduction characteristics; Performing supervision feature transformation processing based on linear discriminant transformation on the dimension reduction features to obtain embedded vectors; and storing the embedded vector and the corresponding change type label thereof to form a feature library.
6. The method for detecting structural change of construction fence and assisting treatment in a multi-mode according to claim 1, wherein the process of calculating the current image depth map, camera internal parameters and the bounding box coordinates to obtain the physical size of the change region comprises: calculating a pixel width and a pixel height based on the bounding box coordinates; Calculating an average depth value of pixels in the boundary box area based on the current image depth map and the boundary box coordinates; Based on the pixel width, the pixel height, the average depth value, and focal length parameters in the camera intrinsic parameters, a physical width, a physical height, and a physical area are calculated.
7. The method for detecting structural change of construction fence and assisting in treatment in multiple modes according to claim 1, wherein the process of processing by adopting a multi-mode large language model based on the structural detection result comprises the following steps: Constructing text prompt information based on the boundary frame coordinates, the change type label and the physical size; Inputting the text prompt information and the current image drawn with the bounding box into the multi-modal large language model; Based on the processing of the multi-modal large language model, natural language text including risk level assessment and maintenance suggestions is output.
8. The method for detecting structural change of construction fence and assisting in multi-mode treatment according to claim 1, wherein after a maintenance work order is generated, a reinspection process is executed based on the maintenance work order, the reinspection process comprises the steps of collecting reinspection images based on a fixed camera after the state of the maintenance work order is changed to be checked, sequentially executing the steps of image registration, feature extraction, depth estimation, change detection and physical size calculation based on the base reference images and the reinspection images, judging that maintenance is effective if structural change is not detected near the boundary frame coordinates, and updating the reinspection images to new base reference images after the maintenance is effective.
9. A system for construction enclosure structural change detection and multi-modal auxiliary treatment, characterized in that it is adapted to implement the method of claim 1, said system comprising: the image acquisition module is used for acquiring an image of the construction enclosure area based on the deployed fixed cameras and acquiring a current image; The image registration module is used for processing the pre-stored reference image and the current image under the sound state of the enclosure by adopting an image registration method to obtain a registered current image; The feature extraction module is used for processing by adopting a feature extraction model based on the base reference image and the registered current image to obtain a reference image feature image and a current image feature image; the depth estimation module is used for processing by adopting a depth estimation model based on the current image feature map to obtain a current image depth map; The change detection module is used for processing by adopting a change detection model based on the reference image feature map, the current image feature map and the current image depth map to obtain the boundary frame coordinates and the change type labels of the enclosing change area; the physical size calculation module is used for carrying out calculation processing based on the current image depth map, the camera internal reference and the boundary frame coordinates to obtain the physical size of the change area; the result construction module is used for constructing a structured detection result based on the boundary frame coordinates, the change type label, the physical size and the current image; the multi-mode large language model module is used for processing and outputting semantic interpretation and maintenance suggestions based on the structural detection result; The work order generation module is used for generating a maintenance work order based on the structural detection result and the semantic interpretation and maintenance suggestion; And the closed loop management module is used for executing a rechecking process based on the base reference image and the new rechecking image after maintenance and updating the base reference image according to the rechecking result.
10. A storage medium having stored thereon a computer program which, when executed by a processor, implements the method of claim 1.

Description

Construction enclosure structural change detection and multi-mode auxiliary treatment method and system Technical Field The invention belongs to the technical field of safety management of constructional engineering, and particularly relates to a method and a system for detecting structural change of a construction enclosure and performing multi-mode auxiliary treatment. Background The construction enclosure is an important safety protection and isolation facility for the urban building construction site, and has the main functions of separating a construction area from an external environment, preventing irrelevant personnel from mistakenly entering the construction site to cause safety accidents, and simultaneously playing a role in beautifying the environment, reducing noise and preventing dust. The stability and integrity of the enclosure is directly related to the safety of surrounding pedestrians and vehicles. At present, supervision of construction barriers mainly depends on the following modes of 1, manual regular inspection, daily or weekly hiking inspection of the barriers of a construction site by a safety person or project manager, visual inspection of whether the problems of breakage, inclination, collapse, doodling, component missing and the like exist, 2, video monitoring manual analysis, installation of a fixed camera at the construction site, transmission of field video to a monitoring center, judgment of whether the barriers are abnormal or not by an attendant through monitoring pictures, 3, partial research adopts means such as frame difference, background modeling, optical flow field estimation and the like or utilizes a traditional machine learning algorithm and a convolutional neural network to carry out object detection or semantic segmentation on a single frame image to try to automatically identify the damaged areas of the barriers, 4, a vibration sensor, an inclination sensor, a switching value sensor and the like are installed on the barrier structure based on a monitoring method of the sensor so as to detect whether the barriers are inclined, whether strong impact exists, whether the barrier doors are opened or not and the like. However, the prior art still has the following problems and defects in practical application that firstly, the labor cost is high, the efficiency is low, and the on-site manual inspection and the video manual staring need to occupy a large amount of manpower, are influenced by fatigue and distraction of personnel, and are difficult to realize continuous and high-precision monitoring. Secondly, semantic understanding capability is lost, the existing image analysis method based on frame difference, optical flow or CNN focuses on pixel level change or abnormal texture, semantic judgment is lacking on whether the change belongs to structural damage of a fence body, the influence of factors such as illumination change, temporary shielding, pedestrian and vehicle interference is easy to occur, and false alarm rate is high. And moreover, the sensor scheme has high false alarm rate and complex deployment and maintenance, the vibration sensor and the like are easily interfered by environmental noise, wind power and mechanical vibration, the hardware layout and long-term maintenance cost is high, and the large-scale application is difficult. Finally, the lack of closed-loop management support is that most systems only stay in the stage of detecting abnormality and alarming, and cannot form an automatic closed loop with links such as maintenance, treatment, effect rechecking and the like, so that complete decision support and flow optimization cannot be provided for safety management. Disclosure of Invention In order to solve the technical problems, the invention provides a method and a system for detecting structural change of a construction fence and performing multi-mode auxiliary treatment, so as to solve the problems in the prior art. In order to achieve the above object, the present invention provides a method for detecting structural change of a construction enclosure and assisting in multi-mode treatment, comprising: acquiring an image of a construction enclosure area, acquiring a current image, and processing the current image and a base reference image by adopting an image registration method to obtain a registered current image; Processing by adopting a feature extraction model based on the base reference image and the registered current image to obtain a reference image feature image and a current image feature image; Based on the current image feature map, processing by adopting a depth estimation model to obtain a current image depth map; Processing the reference image feature map, the current image feature map and the current image depth map by adopting a change detection model to obtain a boundary frame coordinate and a change type label of a surrounding change area; Calculating the current image depth map, the camera internal parameters and the boun