Search

CN-122024091-A - Concrete crack management method and device combining unmanned plane and BIM

CN122024091ACN 122024091 ACN122024091 ACN 122024091ACN-122024091-A

Abstract

The invention provides a concrete crack management method and device combining an unmanned plane and BIM, and relates to the technical field of intelligent detection of civil engineering structures. The method comprises the steps of inspecting the concrete surface by using an unmanned aerial vehicle, carrying out light semantic segmentation by combining fractal features to obtain a crack region, extracting the attribute of each crack to form a crack semantic information set, and simultaneously carrying out multi-view registration and depth sensing three-dimensional positioning and outputting the spatial positions in a local coordinate system and world coordinates of the unmanned aerial vehicle. And converting the coordinates into a BIM model coordinate system to realize accurate alignment with BIM. Creating a crack object graphic element in the BIM, associating semantic information, forming a crack instance data set, and synchronously updating to realize life cycle management. The whole method forms a closed loop flow of 'field acquisition-edge processing-model mapping-information updating-visual management', greatly improves crack detection efficiency and data consistency, and is convenient for continuous monitoring and maintenance management of structural cracks.

Inventors

  • WANG YINGWANG
  • XU ZHEN
  • TIAN YUAN
  • Gu Donglian
  • XIE YUXING

Assignees

  • 北京科技大学

Dates

Publication Date
20260512
Application Date
20251207

Claims (10)

  1. 1. A method of concrete crack management combining an unmanned aerial vehicle and a BIM, the method comprising: s1, carrying out inspection on the surface of a concrete structure by using an unmanned plane to obtain image data and control point data; s2, performing light-weight crack semantic segmentation on the image data by combining fractal feature analysis at an unmanned aerial vehicle end to obtain a segmentation result containing a crack region; S3, extracting crack masks and attribute information of each crack based on the segmentation result of the crack-containing region to obtain a crack semantic information set, wherein the attribute information comprises the size, trend, affiliated components, severity, current state, image acquisition time and attitude and positioning coordinate data of the unmanned aerial vehicle; S4, based on the crack mask and the control point data, carrying out three-dimensional positioning calculation on the crack to obtain the spatial position of the crack and the position of the control point, wherein the three-dimensional positioning calculation comprises a multi-view image registration and a depth sensing method, the multi-view image registration obtains crack point clouds through feature matching and spatial reconstruction, the depth sensing method utilizes laser radar point cloud data to project crack pixels in an image onto a live-action three-dimensional point and calculate the depth and the position of the crack, the spatial position of the crack is expressed through a spatial coordinate system, and the spatial coordinate system comprises a local coordinate system and world coordinates of an unmanned aerial vehicle; S5, converting a space position of the crack into a coordinate system of a BIM model of a corresponding structure, and obtaining a crack coordinate accurately aligned with the BIM three-dimensional model, wherein the coordinate system conversion is realized by a coordinate conversion matrix, the coordinate conversion matrix uses at least two known control points for coordinate calibration and then obtains a coordinate conversion matrix parameter through parameter calculation, the coordinate conversion matrix parameter is a homogeneous transformation matrix, the parameter calculation comprises SVD, a least square method and an iterative optimization algorithm, and the homogeneous transformation matrix is formed by combining a rotation matrix R and a translation vector t; s6, based on the crack coordinates accurately aligned with the BIM three-dimensional model, a component entity of a crack object is created in the BIM, the crack semantic information set is related to a corresponding component entity in the BIM, a crack instance data set in the BIM is obtained, and the crack object integrates the crack geometric form and semantic information in the model in a primitive form; And S7, continuously collecting the semantic information of the crack, and synchronizing the newly added or updated crack information based on the same crack multi-source data merging strategy to obtain a crack example in an updated BIM model, wherein the crack example in the updated BIM model is used for carrying out visual tracking on the crack and closed-loop management on the life cycle of the crack, the same crack multi-source data merging strategy comprises reasonable crack topology assessment and life cycle identification by using a preset symbol or color, and the reasonable crack topology assessment comprises extraction of a merging threshold value based on a box counting fractal dimension based on historical data.
  2. 2. The method for managing concrete cracks by combining an unmanned aerial vehicle and a BIM according to claim 1, wherein the step S1 of using the unmanned aerial vehicle to inspect the surface of the concrete structure to obtain image data and control point data comprises the following steps: S11, carrying out on-site inspection on the surface of a concrete structure by using an unmanned aerial vehicle, wherein the on-site inspection comprises a collection mode of keeping a constant distance of 2-3 meters and parallel skimming the surface of a web plate by using the unmanned aerial vehicle, collecting a crack image and actually measuring the crack position, and the collection mode takes the surface of a member to be detected, of which a bridge web plate is an approximate plane, as an object; S12, dynamically adjusting the flight attitude of the unmanned aerial vehicle after reaching the position near the surface of the member to be detected, and keeping the optical axis of the camera perpendicular to the surface of the member to be detected; s13, realizing close-range shooting through a holder camera, and acquiring an image of the surface of a high-resolution component to be detected to obtain image data and control point data; S14, collecting control point data through a cradle head camera, wherein the control point data are obtained by selecting at least two datum points which are easy to identify on the surface of a concrete structure and measuring the coordinates of the datum points in an unmanned aerial vehicle coordinate system.
  3. 3. The method for managing concrete cracks combining the unmanned aerial vehicle and the BIM according to claim 1, wherein the step S3 is based on the segmentation result of the crack-containing region, extracts a crack mask and attribute information of each crack to obtain a crack semantic information set, wherein the attribute information comprises the size, trend, members, severity, current state, image acquisition time and attitude and positioning coordinate data of the unmanned aerial vehicle, and comprises the following steps: s31, at the unmanned aerial vehicle end, firstly, performing a preprocessing step on the image data to obtain preprocessed image data, wherein the preprocessing step comprises median filtering denoising processing to reduce noise interference; S32, inputting the preprocessed image data into a pre-trained semantic segmentation network, and running in real time in an embedded environment at a video frame rate to obtain a segmentation result, wherein the pre-trained semantic segmentation network is a crack segmentation model based on full convolution, and the crack segmentation model is constructed based on an improved VGG16 or DeepLab architecture; s33, extracting attribute information of each crack according to the segmentation result, wherein the attribute information comprises the form of the crack, an acquisition time stamp, the pose coordinates of the unmanned aerial vehicle, the longitude and latitude coordinates of the shooting position, the altitude and the attitude angle; S34, carrying out fractal feature analysis based on the image data to obtain fractal features, wherein the fractal feature analysis comprises prompt generation based on image local box counting fractal dimension, and the image local box counting fractal dimension comprises covering images with grids under different scales, counting the grid number of pixels containing cracks in each scale and estimating the local fractal dimension according to the grid number; and S35, returning the attribute information and the fractal feature to the ground station through a wireless network.
  4. 4. The method for managing concrete cracks combining a unmanned plane and a BIM according to claim 1, wherein the step S4 is based on the crack mask and the control point data, and performs three-dimensional positioning calculation on the crack to obtain a spatial position of the crack and a position of the control point, the three-dimensional positioning calculation includes a multi-view image registration and a depth sensing method, the multi-view image registration obtains a crack point cloud through feature matching and spatial reconstruction, the depth sensing method projects a crack pixel in an image onto a live-action three-dimensional point by using laser radar point cloud data and calculates a depth and a position of the crack, the spatial position of the crack is expressed by a spatial coordinate system, the spatial coordinate system includes a local coordinate system of the unmanned plane and world coordinates, and the method includes: s41, extracting position information, an IMU attitude angle and an unmanned aerial vehicle position coordinate from the crack semantic information set to obtain a space geometric position of the crack on the structural surface, wherein the space geometric position is expressed through a pixel coordinate system; s42, performing feature matching and space reconstruction by adopting multi-view image registration to obtain a crack point cloud, wherein the multi-view image registration comprises feature matching and space reconstruction of multi-view images, and the crack point cloud comprises a point set for representing the space distribution of cracks; S43, acquiring laser radar point cloud data through a depth sensing method, obtaining the relation of the point cloud data projected to the space geometric position of the crack on the surface of the structure, and obtaining the elevation of the plane of the component and the position of the center of the crack in the image, wherein the depth sensing method utilizes the laser radar point cloud data for projection; s44, projecting the position coordinates of the unmanned aerial vehicle to a plane where the component is located, and obtaining a coordinate point of the center of the image on the surface of the structure based on the elevation of the plane where the component is located; S45, obtaining an actual horizontal distance corresponding to the pixel offset through conversion of an internal camera parameter and a height according to the pixel offset of the position of the crack center in the image relative to the image center, wherein the pixel offset is based on the image center, and the internal camera parameter comprises a parameter for conversion of an imaging scale; And S46, carrying out superposition calculation on the actual horizontal distance and the space geometric position of the crack on the structural surface to obtain the space position of the crack, wherein the superposition calculation comprises adding the actual horizontal distance to the space geometric position, the space position of the crack is expressed in a space coordinate system, the space coordinate system expression comprises a double-coordinate result for positioning and expression, the space coordinate system comprises a local coordinate system and world coordinates of the unmanned aerial vehicle, and the double-coordinate result takes the structural surface as a reference to ensure the position consistency.
  5. 5. The method for managing concrete cracks combining a unmanned plane and a BIM according to claim 1, wherein the spatial position of the crack in S5 is subjected to coordinate system conversion, the spatial coordinate system is converted to a BIM model coordinate system of a corresponding structure, a crack coordinate aligned with the BIM three-dimensional model is obtained, the coordinate system conversion is realized by a coordinate conversion matrix, the coordinate conversion matrix uses at least two known control points to perform coordinate calibration and then obtains a coordinate conversion matrix parameter through parameter calculation, the coordinate conversion matrix parameter is a homogeneous transformation matrix, the parameter calculation includes SVD, a least square method and an iterative optimization algorithm, the homogeneous transformation matrix is formed by combining a rotation matrix R and a translation vector t, and the method comprises the following steps: s51, extracting coordinate values of the control points under an unmanned plane coordinate system and a BIM model coordinate system based on the control point data; S52, based on coordinate values of control points in an unmanned plane coordinate system and a BIM model coordinate system, obtaining a coordinate conversion matrix through rigid transformation and parameter calculation, wherein the coordinate conversion matrix adopts a homogeneous transformation form and comprises conversion parameters, the conversion parameters comprise a rotation matrix R, a translation vector t and optional unified scale factors, and the parameter calculation comprises singular value decomposition SVD, a least square method and an iterative optimization algorithm; And S53, applying two-stage transformation to all space points or point cloud coordinates of the crack based on the coordinate transformation matrix to obtain crack coordinates accurately aligned with the BIM three-dimensional model, wherein the two-stage transformation comprises the steps of calling a Gaussian projection algorithm to transform the geographic coordinates acquired by the unmanned aerial vehicle into a plane rectangular coordinate system, and then transforming the plane rectangular coordinate system into a BIM model coordinate system based on the coordinate transformation matrix.
  6. 6. The method for concrete crack management combining a drone and a BIM according to claim 1, wherein the creating of the component entities of the crack object in the BIM model based on the crack coordinates precisely aligned with the BIM three-dimensional model in S6, associating the crack semantic information set to the corresponding component entities in the BIM model, obtaining a crack instance data set in the BIM model, the crack object integrating the crack geometry and semantic information in the form of primitives in the model, includes: S61, automatically creating a crack object based on the crack coordinates accurately aligned with the BIM three-dimensional model to obtain a crack example in the BIM model, wherein the automatically creating the crack object comprises inserting a predefined crack primitive which is in the form of an elongated folding line segment, a curve or a two-dimensional symbol and is used for storing in the model in the form of a parameterized primitive and establishing data association with the surface of the structural member where the crack primitive is located, wherein the predefined crack primitive is inserted into the corresponding member surface at the determined crack space position as a geometric representation form with the size and the position consistent with the actual crack; s62, inputting a crack semantic information set into parameter fields of a crack object, wherein the attribute parameters comprise a crack number, a discovery time, a spatial position, a geometric dimension, a severity and a current state, and the geometric dimension comprises a length, an average width, a maximum width and a depth; and S63, establishing a visual mark for the crack object by adopting a preset symbol or color to obtain differentiated display of different cracks and states thereof, wherein the preset symbol or color comprises a pattern set according to a life cycle mark, and the life cycle mark comprises a new discovery, continuous expansion, stability, repaired and/or severity level.
  7. 7. The method for managing concrete cracks combining a unmanned plane and a BIM according to claim 1, wherein the continuously collecting the semantic information of the cracks in S7 and synchronizing the newly added or updated crack information based on the same multi-source data combining strategy of the cracks to obtain a crack instance in an updated BIM model, wherein the crack instance in the updated BIM model is used for visually tracking the cracks and performing closed-loop management of the life cycle of the cracks, the same multi-source data combining strategy of the cracks includes reasonable evaluation of the topology of the cracks and performing life cycle identification by using a predetermined symbol or color, the reasonable evaluation of the topology of the cracks includes extracting a combining threshold based on a box count fractal dimension based on historical data, and the method includes: s71, continuously collecting crack semantic information from the inspection and model maintenance flow, and adding newly obtained crack records into a crack instance data set in the BIM; S72, acquiring a box counting fractal dimension of a newly acquired crack record, and judging whether a combinable object exists in a crack instance data set of a newly added or updated crack in an existing BIM model by combining the position, the length and the trend information of a central point, wherein the combinable object refers to a crack instance with similar attribute of the crack instance data set in the existing BIM model, and the box counting fractal dimension, the length and the trend information are used for calculating a life cycle identifier; And S73, if the combinable object exists, updating parameters of the combinable object, adding a life cycle identifier in the history attribute once, and if the combinable object does not exist, building a crack instance newly so as to ensure that the same crack corresponds to only one object.
  8. 8. A concrete crack management device combining an unmanned aerial vehicle and a BIM for realizing the concrete crack management method combining an unmanned aerial vehicle and a BIM according to any one of claims 1 to 7, wherein the device comprises: the inspection module is used for inspecting the surface of the concrete structure by using the unmanned aerial vehicle to obtain image data and control point data; the semantic segmentation module is used for executing light-weight crack semantic segmentation on the unmanned aerial vehicle end by combining the image data with fractal feature analysis to obtain a segmentation result containing a crack region; The information extraction module is used for extracting crack masks and attribute information of each crack based on the segmentation result of the crack-containing region to obtain a crack semantic information set, wherein the attribute information comprises the size, trend, the belonged component, severity, the current state, the image acquisition time and the posture and positioning coordinate data of the unmanned aerial vehicle; The three-dimensional positioning module is used for carrying out three-dimensional positioning calculation on the crack based on the crack mask and the control point data to obtain the spatial position of the crack and the position of the control point, the three-dimensional positioning calculation comprises a multi-view image registration method and a depth sensing method, the multi-view image registration method is used for obtaining a crack point cloud through feature matching and spatial reconstruction, the depth sensing method is used for projecting crack pixels in an image onto a real scene three-dimensional point by utilizing laser radar point cloud data and calculating the depth and the position of the crack, the spatial position of the crack is expressed through a spatial coordinate system, and the spatial coordinate system comprises a local coordinate system and world coordinates of the unmanned aerial vehicle; The coordinate alignment module is used for converting a space position of the crack into a coordinate system of a BIM model of a corresponding structure, obtaining a crack coordinate aligned with the BIM three-dimensional model accurately, wherein the coordinate system conversion is realized by a coordinate conversion matrix, the coordinate conversion matrix uses at least two known control points for coordinate calibration and then obtains a coordinate conversion matrix parameter through parameter calculation, the coordinate conversion matrix parameter is a homogeneous transformation matrix, the parameter calculation comprises SVD, a least square method and an iterative optimization algorithm, and the homogeneous transformation matrix is formed by combining a rotation matrix R and a translation vector t; The BIM integration module is used for creating a component entity of a crack object in the BIM based on the crack coordinate accurately aligned with the BIM three-dimensional model, associating the crack semantic information set to a corresponding component entity in the BIM to obtain a crack instance data set in the BIM, and integrating the crack geometric form and semantic information of the crack object in the model in a graphic primitive form; The life cycle module is used for continuously collecting the semantic information of the cracks and synchronizing the newly added or updated crack information based on the same crack multi-source data merging strategy to obtain a crack example in an updated BIM model, wherein the crack example in the updated BIM model is used for carrying out visual tracking on the cracks and closed-loop management on the life cycle of the cracks, the same crack multi-source data merging strategy comprises reasonable crack topology assessment and life cycle identification by using a preset symbol or color, and the reasonable crack topology assessment comprises extraction of a merging threshold value based on box counting fractal dimension based on historical data.
  9. 9. A concrete crack management device combining a drone and a BIM, wherein the concrete crack management device combines the drone and the BIM, and wherein the memory has stored thereon computer readable instructions that, when executed by the processor, implement the method of any one of claims 1 to 7.
  10. 10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a program code, which is callable by a processor for executing the method according to any one of claims 1 to 7.

Description

Concrete crack management method and device combining unmanned plane and BIM Technical Field The invention relates to the technical field of intelligent detection of civil engineering structures, in particular to a concrete crack management method and device combining an unmanned plane and a BIM. Background In the long-term service process of engineering structures such as concrete bridges and buildings, the construction process, the environmental influence or the repeated load effect often causes diseases such as cracks and flaking on the surfaces of the components, and the safety and the durability of the structure are seriously affected. The crack on the structural surface is detected and identified timely and accurately, and the method has important significance for making maintenance measures, eliminating potential safety hazards and prolonging the service life. Traditional crack inspection mainly relies on manual close-range inspection and photographing record, and is low in efficiency, poor in accessibility and affected by human factors, and acquired image data is difficult to manage in a standardized mode and store for a long time. Along with the development of unmanned aerial vehicle technology, adopting unmanned aerial vehicle to carry high definition digtal camera and carry out bridge or building crack inspection has become one of the mainstream modes. The unmanned aerial vehicle has the characteristics of high efficiency and flexibility, can cover the part which is difficult to reach by manpower, rapidly collects the surface images of a large-scale structure, and provides convenience conditions for crack detection. In the operation and maintenance stage of the engineering structure, comprehensive and accurate management of disease information such as cracks is needed. BIM technology can integrate various information of the whole life cycle of the structure, and is an effective means for supporting long-term operation and maintenance management of the infrastructure. The crack detection result obtained by unmanned aerial vehicle inspection is associated with the BIM three-dimensional model, so that positioning marking and information association of the crack in a three-dimensional space can be realized, and visual basis is provided for subsequent maintenance decision. However, the existing linkage of unmanned aerial vehicle image data and BIM model still has many challenges, namely, the traditional image and model registration method mostly depends on crack texture characteristics or manually marked datum points, the surface texture of concrete structures such as bridges is single, the characteristics are insufficient, and automatic registration and positioning based on image characteristics are difficult to realize. Some researches try to use GPS information carried by unmanned aerial vehicle images for crack position positioning, but few aspects of field application and accuracy verification exist, and no mature scheme is formed yet. In addition, a large amount of high-definition images acquired by unmanned aerial vehicles in the prior art are usually required to be transmitted back to the ground for offline processing, and the time consumption for data transmission and processing is long, so that the real-time identification and feedback of cracks in the inspection process can not be realized. Therefore, a new technical scheme is needed, the mobility and the on-site computing capability of the unmanned plane platform can be fully utilized, the real-time detection and the accurate positioning of the cracks are realized, and the detection result is automatically fused into the BIM operation and maintenance model for visual management. The efficiency and informatization degree of crack inspection are greatly improved, and a powerful support is provided for maintenance decision of the structure. Disclosure of Invention In addition, a large amount of high-definition images acquired by unmanned aerial vehicles in the prior art usually need to be transmitted back to the ground for offline processing, the data transmission and processing are long in time consumption, the immediate identification and feedback of cracks in the inspection process can not be realized, and the detection result is automatically fused into a BIM operation and maintenance model for visual management. The technical scheme is as follows: In one aspect, there is provided a concrete crack management method combining an unmanned aerial vehicle and a BIM, the method being implemented by a concrete crack management apparatus combining the unmanned aerial vehicle and the BIM, the method comprising: s1, carrying out inspection on the surface of a concrete structure by using an unmanned plane to obtain image data and control point data; s2, performing light-weight crack semantic segmentation on the image data by combining fractal feature analysis at an unmanned aerial vehicle end to obtain a segmentation result containing a crack regio