CN-121982642-A - Construction machine detection method and device, storage medium and electronic equipment

CN121982642ACN 121982642 ACN121982642 ACN 121982642ACN-121982642-A

Abstract

The invention discloses a detection method and device of construction machinery, a storage medium and electronic equipment. The method comprises the steps of utilizing a multi-source sensor to collect data of a construction area to obtain multi-source detection data, generating target point cloud data of the construction area based on the multi-source detection data, processing the target point cloud data to obtain time sequence voxel data, wherein the time sequence voxel data comprise voxels obtained by stacking a plurality of voxels according to a time stamp sequence, inputting the time sequence voxel data into a target neural network to obtain a segmentation point cloud, the target neural network is used for evaluating category labels of each voxel, the segmentation point cloud comprises a semantic point set, the semantic point set comprises a point set mapped with the category labels of each voxel in the time sequence voxel data, and determining the action type of a construction machine based on the segmentation point cloud. The invention solves the technical problem of low accuracy of the construction machinery action recognition in the related technology.

Inventors

YI SHUWEI
Lang Yecun
YANG YABIN
JIAO ZHENGGUO
LUO CHAOLONG
LIU BOYU
HE ZIQING
LI WEI
CAI QING
MA FENG
FANG HUALIN
ZHOU KAI
LI HUA
TAO KUN
LIU JUNJIE
MANG XIUWEI

Assignees

国网北京市电力公司

Dates

Publication Date: 20260505
Application Date: 20260127

Claims (10)

1. A method of detecting a construction machine, comprising: The method comprises the steps of utilizing a multi-source sensor to acquire data of a construction area to obtain multi-source detection data, wherein the multi-source sensor comprises at least one of a multi-line laser radar, an industrial camera and an inertial measurement unit; generating target point cloud data of the construction area based on the multi-source detection data, and processing the target point cloud data to obtain time sequence voxel data, wherein the time sequence voxel data comprises voxels obtained by stacking a plurality of voxels according to a time stamp sequence, and the plurality of voxels comprise voxels obtained by voxel coding the target point cloud data; inputting the time sequence voxel data into a target neural network to obtain a partition point cloud, wherein the target neural network is used for evaluating the class label of each voxel, the partition point cloud comprises a semantic point set, and the semantic point set comprises a point set mapped with the class label of each voxel in the time sequence voxel data; And determining the action type of the construction machine based on the segmentation point cloud.
2. The detection method according to claim 1, wherein determining an action type of a construction machine based on the split point cloud includes: clustering and shape constraint are carried out on the semantic point set in the segmentation point cloud to obtain an instance set, wherein the instance set comprises a plurality of instances; Adopting a target strategy to correlate adjacent frame examples in the example set to obtain an example sequence, wherein the target strategy comprises at least one of Kalman filtering and Hungary matching; Based on the example sequence, a type of action of the work machine is determined.
3. The method of detection of claim 2, wherein determining the type of action of the work machine based on the sequence of instances comprises: Determining a working radius dynamic boundary of the construction machine based on the example sequence, wherein the working radius dynamic boundary represents a space region where the tail end of the construction machine is reachable under the current gesture; extracting corresponding point cloud data from the tail end area of the dynamic boundary of the working radius to obtain boundary point cloud data; Determining the material volume of the end region of the working radius dynamic boundary based on the boundary point cloud data; and determining the action type of the construction machine based on the working radius dynamic boundary and the material volume.
4. The detection method according to claim 2, wherein clustering and shape constraint are performed on the semantic point set in the segmentation point cloud to obtain an instance set, including: based on the semantic point set in the partition point cloud, respectively extracting the point set corresponding to each type of label in the partition point cloud to obtain a plurality of types of construction machine point sets; Clustering each type of construction machinery point set by adopting a target clustering strategy to obtain an initial instance cluster set, wherein the target clustering strategy comprises an Euclidean clustering strategy; and screening the examples in the initial example cluster set based on a preset shape constraint rule to obtain the example set.
5. The detection method according to claim 1, wherein the multi-source detection data includes three-dimensional point cloud data of the construction area, image data of the construction area, and attitude data and acceleration data of the multi-source sensor, and generating target point cloud data of the construction area based on the multi-source detection data includes: performing time stamp alignment and posture correction on the multi-source detection data to obtain target multi-source data; performing external parameter calibration on the target multi-source data to obtain calibrated target multi-source data; And converting the calibrated target multi-source data into a unified engineering coordinate system to obtain the target point cloud data.
6. The detection method according to claim 1, wherein the processing the target point cloud data to obtain time-series voxel data further comprises: Performing ground separation processing on the target point cloud data to obtain first point cloud data; Performing normal estimation and curvature calculation on the first point cloud data to obtain a calculation result; And carrying out voxel coding on the first point cloud data to obtain a coding result, and determining the time sequence voxel data based on the calculation result and the coding result, wherein the coding result comprises a plurality of voxels.
7. The detection method according to claim 1, wherein the target neural network adopts a U-shaped structure with symmetric coding ends and decoding ends, the coding ends are alternately composed of a plurality of sparse three-dimensional convolution layers and downsampling modules, and are used for gradually aggregating local and global spatial features, the decoding ends restore spatial resolution step by step through sparse upsampling layers, feature mapping of corresponding layers of the coding ends is fused with features of the decoding ends through jump connection, the time sequence voxel data is input into the target neural network, and a segmentation point cloud is obtained, and the detection method comprises: inputting the time sequence voxel data into the target neural network, and outputting a classification result of each voxel and an uncertainty estimation value, wherein the classification result is used for indicating the probability distribution of the class label of each voxel, and the uncertainty estimation value is used for representing the confidence of the probability distribution; And mapping the classification result of each voxel to the target point cloud data to obtain the segmentation point cloud.
8. A detection device for a construction machine, comprising: The multi-source sensor is used for acquiring data of a construction area by utilizing the multi-source sensor to obtain multi-source detection data, wherein the multi-source sensor comprises at least one of a multi-line laser radar, an industrial camera and an inertia measurement unit; The first processing unit is used for generating target point cloud data of the construction area based on the multi-source detection data and processing the target point cloud data to obtain time sequence voxel data, wherein the time sequence voxel data comprises voxels obtained by stacking a plurality of voxels according to a time stamp sequence, and the plurality of voxels comprise voxels obtained by voxel coding the target point cloud data; The second processing unit is used for inputting the time sequence voxel data into a target neural network to obtain a division point cloud, wherein the target neural network is used for evaluating the category label of each voxel, the division point cloud comprises a semantic point set, and the semantic point set comprises a point set mapped with the category label of each voxel in the time sequence voxel data; and the determining unit is used for determining the action type of the construction machine based on the segmentation point cloud.
9. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and wherein the computer program, when executed, controls a device in which the computer-readable storage medium is located to perform the detection method of the construction machine according to any one of claims 1 to 7.
10. An electronic device comprising one or more processors and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of detection of a work machine of any of claims 1-7.

Description

Construction machine detection method and device, storage medium and electronic equipment Technical Field The present invention relates to the field of computer vision and machine learning, and in particular, to a method and apparatus for detecting construction machinery, a storage medium, and an electronic device. Background There are multiple types of mechanical equipment on the construction site, the operation range is wide, the movement track is complex, and the mechanical equipment is influenced by factors such as dust, illumination change, shielding and multi-mechanical cooperative operation in the construction environment, and the monitoring and identification of the mechanical equipment have higher difficulty. The traditional monitoring method based on the two-dimensional video depends on the color and texture characteristics of the image, is easy to identify errors when insufficient illumination, view angle shielding or similar object appearance exist, and is difficult to accurately estimate the spatial position and the operation state of the machine. With the development of three-dimensional sensors such as laser radar, the target identification method based on point cloud data can directly acquire the position and size information of the machine in a three-dimensional space. However, most of the existing construction machinery detection methods based on the point cloud are semantic segmentation and target detection of single-frame point cloud, motion information in continuous time sequence point cloud cannot be fully utilized, and accuracy of occlusion recovery, track association and operation state identification in a dynamic environment is insufficient. Meanwhile, the conventional action recognition method mostly adopts a state machine model with a fixed threshold value, only depends on the mechanical centroid track or the key point gesture change to judge, and does not combine the special working radius and material change characteristics of the construction machinery, so that the reliability of action recognition in a complex construction scene is not high. In view of the above problems, no effective solution has been proposed at present. Disclosure of Invention The embodiment of the invention provides a detection method and device of construction machinery, a storage medium and electronic equipment, which are used for at least solving the technical problem of low precision of construction machinery action recognition in the related technology. According to one aspect of the embodiment of the invention, a detection method of a construction machine is provided, wherein the detection method comprises the steps of acquiring data of a construction area by utilizing a multi-source sensor to obtain multi-source detection data, wherein the multi-source sensor comprises at least one of a multi-line laser radar, an industrial camera and an inertia measurement unit, generating target point cloud data of the construction area based on the multi-source detection data, processing the target point cloud data to obtain time sequence voxel data, the time sequence voxel data comprises voxels obtained by stacking a plurality of voxels according to a time stamp sequence, the voxels comprise voxels obtained by voxel coding the target point cloud data, and the time sequence voxel data is input into a target neural network to obtain a division point cloud, wherein the target neural network is used for evaluating category labels of each voxel, the division point cloud comprises a semantic point set, the semantic point set comprises category labels of each voxel mapped in the time sequence voxel, and the type of the construction machine is determined based on the division point cloud. Further, determining the action type of the construction machine based on the segmentation point cloud comprises the steps of clustering a semantic point set in the segmentation point cloud and restraining the shape to obtain an instance set, wherein the instance set comprises a plurality of instances, and associating adjacent frame instances in the instance set by adopting a target strategy to obtain an instance sequence, wherein the target strategy comprises at least one of Kalman filtering and Hungary matching, and determining the action type of the construction machine based on the instance sequence. Further, determining the action type of the construction machine based on the instance sequence comprises determining a working radius dynamic boundary of the construction machine based on the instance sequence, wherein the working radius dynamic boundary represents a space region where the tail end of the construction machine can reach under the current gesture, extracting corresponding point cloud data from the tail end region of the working radius dynamic boundary to obtain boundary point cloud data, determining the material volume of the tail end region of the working radius dynamic boundary based on the boundary point cloud data,