CN-122023885-A - Target detection method, device, equipment and storage medium based on color map and depth map

CN122023885ACN 122023885 ACN122023885 ACN 122023885ACN-122023885-A

Abstract

The application discloses a target detection method, a device, equipment and a storage medium based on a color image and a depth image, which relate to the technical field of safety monitoring, wherein an image acquisition device synchronously acquires a platform exit color image and a depth image, and after preprocessing, all-target detection is respectively executed; the method comprises the steps of obtaining a first target detection result through a lightweight model, converting a depth image into point cloud data, detecting to obtain a second target detection result, inputting the first target detection result and the second target detection result into a secondary verification model, carrying out consistency verification through a full-target detection large model, realizing class identification by combining an open set classification model, carrying out differential verification logic aiming at targets with different sizes, directly alarming the large target and alarming the small target only when the small target meets a matching degree threshold value.

Inventors

WANG XIANFEI
PENG SHULIN
Ling Guangzheng
ZHONG LIN
LIU ZHONGBIN
CHEN WENTONG
OUYANG ZILING

Assignees

佳都科技集团股份有限公司
广州佳都智通科技有限公司
广东华之源信息工程有限公司
广州华佳软件有限公司
广州佳都城轨智慧运维服务有限公司

Dates

Publication Date: 20260512
Application Date: 20260113

Claims (10)

1. The target detection method based on the color map and the depth map is characterized by comprising the following steps of: Synchronously acquiring a color image and a depth image of a target object through an image acquisition device; Executing first target detection processing on the color image to obtain a first target detection result; Converting the depth image into point cloud data information, and executing second target detection processing on the point cloud data information to obtain a second target detection result; inputting the first target detection result and the second target detection result into a trained secondary verification model, performing target consistency verification and category identification through the secondary verification model, and outputting a verification result; And generating target detection alarm information according to the verification result, carrying out target category identification through the secondary verification model based on the alarm information, and outputting a target category identification result.
2. The method for detecting an object based on a color map and a depth map according to claim 1, wherein the performing a first object detection process on the color image to obtain a first object detection result includes: performing first target detection processing on the color image by adopting a target detection model, and identifying a target area image of the target object in the color image; extracting target area boundary frame coordinates of the target area image to obtain target area boundary frame coordinate data; And taking the coordinate data of the boundary frame of the target area as the first target detection result.
3. The method for detecting an object based on a color map and a depth map according to claim 1, wherein the converting the depth image into point cloud data information, and performing a second object detection process on the point cloud data information to obtain a second object detection result, includes: Converting the depth image into three-dimensional point cloud data according to internal and external parameters of the image acquisition equipment, and identifying a three-dimensional point cloud data target area image comprising the target object according to the three-dimensional point cloud data; Intercepting the three-dimensional point cloud data of the identified three-dimensional point cloud data target area image, and performing three-dimensional point cloud filtering processing based on the intercepted three-dimensional point cloud data target area image; Extracting the three-dimensional point cloud data comprising the target object through a three-dimensional point cloud segmentation algorithm based on the three-dimensional point cloud data target area image after the three-dimensional point cloud filtering processing to obtain a target object three-dimensional coordinate corresponding to the three-dimensional point cloud data; and taking the three-dimensional coordinates of the target object as the second target detection result.
4. The color map and depth map-based object detection method according to claim 1, wherein the secondary verification model includes a full object detection large model and an open set classification model; inputting the first target detection result and the second target detection result into a trained secondary verification model, and performing target consistency verification and category identification through the secondary verification model, wherein the method comprises the following steps of: inputting the color image and the first target detection result into the full-target detection large model to execute third target detection processing to obtain a third target detection result; Performing target consistency matching degree verification on the first target detection result and the third target detection result to obtain a verification result, and generating alarm information when the verification result meets a preset condition; Judging the image size of the target area based on the second target detection result, executing differential logic verification on the target area images with different image sizes, obtaining a verification result, and generating alarm information when the verification result meets a preset condition; Extracting a target area image corresponding to the third target detection result from the color image and extracting a target area image corresponding to the second target detection result from the color image based on the alarm information, inputting the target area image into the open set classification model, and extracting visual feature vectors of the target area image through the open set classification model; And performing similarity matching on the visual feature vector and a preset target category text vector, and obtaining a target category identification result based on the similarity matching result.
5. The color map and depth map-based object detection method according to claim 4, wherein the full object detection large model is trained by: acquiring a public scene color image dataset, and performing open word class labeling on a target object in the public scene color image dataset to obtain first labeling data with an open word class label; inputting the first annotation data serving as a text prompt word and a corresponding color image into an initial full-target detection large model, and performing first-stage pre-training to obtain a pre-training model with open word detection capability; acquiring a real scene color image dataset of an applicable scene, labeling the target object of the real scene color image dataset in the applicable scene, and replacing an open word class label of the target object with a single prompt word to obtain second labeling data; And carrying out second-stage adjustment on the pre-training model by using the second labeling data to obtain a full-target detection large model.
6. The color map and depth map-based object detection method of claim 4, wherein said performing a differential logic check on said object region images of different image sizes comprises: When the size of the target area image is larger than a preset large-size threshold, judging that the target area image is a large target, and generating alarm information according to the three-dimensional coordinate information of the second target detection result; And when the size of the target area image is smaller than a preset large-size threshold, judging that the target area image is a small target, calculating the threshold of the 2D target area boundary box of the second target detection result and the threshold of the 2D target area boundary box of the third target detection result, and generating alarm information when the threshold meets a preset condition.
7. The color map and depth map-based object detection method according to claim 1, further comprising, after synchronously acquiring the color image and the depth image of the object by the image acquisition device: Performing illumination equalization and noise suppression processing on the color image; D2C alignment and cavity filling processing are carried out on the depth image.
8. An object detection device based on a color map and a depth map, comprising: the image acquisition module is used for synchronously acquiring a color image and a depth image of the target object through the image acquisition equipment; The first detection module is used for executing first target detection processing on the color image to obtain a first target detection result; The second detection module is used for converting the depth image into point cloud data information, and executing second target detection processing on the point cloud data information to obtain a second target detection result; the secondary verification module is used for inputting the first target detection result and the second target detection result into a trained secondary verification model, carrying out target consistency verification and category identification through the secondary verification model, and outputting a verification result; And the alarm generation module is used for generating target detection alarm information according to the verification result, carrying out target category identification through the secondary verification model based on the alarm information, and outputting a target category identification result.
9. An electronic device comprising a processor and a memory, wherein the memory has stored therein computer readable instructions which when executed by the processor implement the steps of the color map and depth map based object detection method as claimed in any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the color map and depth map based object detection method according to any of claims 1-7.

Description

Target detection method, device, equipment and storage medium based on color map and depth map Technical Field The application relates to the technical field of safety monitoring, in particular to a target detection method, device, equipment and storage medium based on a color map and a depth map. Background In inter-city railway operation, a platform is taken as a key transition area between a train and a platform door, safety monitoring is directly related to driving safety and passenger guarantee, targets such as retained passengers and left foreign matters are required to be detected in time to avoid accidents, the current mainstream platform-exiting safety detection technology has a remarkably short board, a detection system based on laser correlation spans up to 200 meters and is easily influenced by tiny vibration to cause abnormal signal reception, a large number of false alarms are generated, alarm information cannot be visually presented, manual field checking is required, efficiency is low, in a vision-based detection scheme, although a background modeling method for traditional image processing can detect any target in real time, the target type cannot be distinguished, the targets are easily interfered by ambient light and weather changes, background modeling difficulty is high, robustness is insufficient, and a target detection method based on deep learning can intuitively output alarm information, but has strong data dependence and poor generalization capability, is difficult to adapt to platform-exiting scenes of different structures, the targets are required to be pre-defined, various foreign matters on the platform cannot be detected, and the recall rate of detection is low. In addition, the platform is usually in an outdoor environment, direct sunlight and complex illumination can influence imaging stability, small targets (such as objects smaller than 10cm multiplied by 10 cm) are prone to missed detection, detection reliability is further reduced, comprehensive requirements of platform target detection on instantaneity, high recall rate, high precision and generalization capability are difficult to meet in the prior art, and a collaborative detection scheme for fusing multi-mode data and intelligent algorithms is needed to break the technical bottlenecks. Disclosure of Invention The embodiment of the application provides a target detection method, device, equipment and storage medium based on a color map and a depth map, which realize the problem that the real-time performance and the precision of the traditional technology are difficult to be compatible by using a layered architecture of multi-mode acquisition, light-weight front-end detection and accurate back-end verification. In a first aspect, an embodiment of the present application provides a target detection method based on a color map and a depth map, including the following steps: Synchronously acquiring a color image and a depth image of a target object through an image acquisition device; Executing first target detection processing on the color image to obtain a first target detection result; Converting the depth image into point cloud data information, and executing second target detection processing on the point cloud data information to obtain a second target detection result; inputting the first target detection result and the second target detection result into a trained secondary verification model, performing target consistency verification and category identification through the secondary verification model, and outputting a verification result; And generating target detection alarm information according to the verification result, carrying out target category identification through the secondary verification model based on the alarm information, and outputting a target category identification result. Further, the performing a first object detection process on the color image to obtain a first object detection result includes: performing first target detection processing on the color image by adopting a target detection model, and identifying a target area image of the target object in the color image; extracting target area boundary frame coordinates of the target area image to obtain target area boundary frame coordinate data; And taking the coordinate data of the boundary frame of the target area as the first target detection result. Further, the converting the depth image into point cloud data information, and executing a second target detection process on the point cloud data information to obtain a second target detection result, including: Converting the depth image into three-dimensional point cloud data according to internal and external parameters of the image acquisition equipment, and identifying a three-dimensional point cloud data target area image comprising the target object according to the three-dimensional point cloud data; Intercepting the three-dimensional point cloud data of the identified three