CN-117291983-B - Goods shelf position detection method, device, equipment and storage medium

CN117291983BCN 117291983 BCN117291983 BCN 117291983BCN-117291983-B

Abstract

The application discloses a method, a device, equipment and a storage medium for detecting the position of a goods shelf, wherein a three-dimensional point cloud image shot by a depth camera is acquired, a region where a goods shelf is located is detected from the three-dimensional point cloud image based on a target detection algorithm of depth learning, goods shelf legs are segmented from the region where the goods shelf is located based on a target segmentation algorithm of depth learning, the center position of the goods shelf and the gesture of the goods shelf are calculated based on the point cloud coordinates in the goods shelf legs.

Inventors

LI JIAXING
LUO ZENGHUI
LAI ZHILIN
YANG XIAODONG

Assignees

广州赛特智能科技有限公司

Dates

Publication Date: 20260505
Application Date: 20231030

Claims (8)

1. A method for detecting a position of a shelf, comprising: acquiring a three-dimensional point cloud image shot by a depth camera; Detecting the region where the goods shelf is located from the three-dimensional point cloud image by a target detection algorithm based on deep learning; dividing shelf legs from the region where the shelf is located by a target dividing algorithm based on deep learning; calculating the center position of the goods shelf and the gesture of the goods shelf based on the point cloud coordinates in the goods shelf legs; Wherein before the deep learning-based target segmentation algorithm segments the shelf legs from the region where the shelf is located, the method further comprises: Performing point cloud completion on a target in an area where the goods shelf is located; The performing point cloud completion on the target in the area where the goods shelf is located includes: converting the irregular point cloud data in the area where the goods shelf is located into normalized three-dimensional grid data; Inputting the three-dimensional grid data into a three-dimensional convolution network for processing to obtain three-dimensional grid data after convolution processing; inversely converting the three-dimensional grid data after convolution processing into point cloud data; sampling the point cloud data after the reverse conversion to obtain rough point cloud data; and shifting the points in the rough point cloud data to generate new points to perform point cloud completion on the targets in the area where the goods shelf is located.
2. The shelf position detection method according to claim 1, wherein the detection of the region where the shelf is located from the three-dimensional point cloud image by a target detection algorithm based on deep learning comprises: dividing all points in the three-dimensional point cloud image into a plurality of three-dimensional voxels with the same size; Extracting the characteristics of points in each three-dimensional voxel to obtain a characteristic matrix formed by characteristic vectors of each three-dimensional voxel; and carrying out classification detection and position regression based on the feature matrix to obtain a three-dimensional boundary box of the area where the goods shelf is located.
3. The shelf position detection method according to claim 2, wherein performing feature extraction on points within each of the three-dimensional voxels to obtain a feature matrix composed of feature vectors of each of the three-dimensional voxels, comprises: Calculating the coordinate mean value of all points in the three-dimensional voxel as the centroid coordinates of the three-dimensional voxel aiming at each three-dimensional voxel; Calculating the difference value between the coordinates of each point in the three-dimensional voxel and the coordinates of the mass center of the three-dimensional voxel to obtain the relative coordinates of each point in the three-dimensional voxel and the mass center; Splicing the vector representation of the point coordinates with the vector representation of the relative coordinates to obtain a position vector of each point in the three-dimensional voxel; Inputting the position vector of the point in the three-dimensional voxel into a fully connected network, and mapping the position vector of the point in the three-dimensional voxel by the fully connected network to obtain a mapping characteristic; Inputting the mapping characteristics into a residual convolution network, and carrying out characteristic mining on the mapping characteristics by the residual convolution network to obtain characteristic vectors of the three-dimensional voxels; and combining the feature vectors of the three-dimensional voxels to form a feature matrix.
4. The method for detecting the position of a shelf according to claim 2, wherein the step of performing classification detection and position regression based on the feature matrix to obtain a three-dimensional bounding box of an area where the shelf is located comprises: Inputting the feature matrix into a region generation network, processing the feature matrix by the region generation network, and carrying out classification detection and position regression on the targets in the three-dimensional point cloud image to obtain a three-dimensional boundary box of the region where the goods shelf is located.
5. The shelf location detection method according to any one of claims 1-4, wherein a deep learning based object segmentation algorithm segments shelf legs from an area where the shelf is located, comprising: Extracting local features of all points in the area where the goods shelf is located to obtain the local features of the area where the goods shelf is located; Pooling the local features in each dimension to obtain global features of the region where the goods shelf is located; Splicing the local features and the global features in dimensions to obtain fusion features; And dividing a shelf leg from the region of the shelf based on the fusion characteristic.
6. A shelf position detection apparatus comprising: The point cloud image acquisition module is used for acquiring a three-dimensional point cloud image shot by the depth camera; The goods shelf area detection module is used for detecting the area where the goods shelf is located from the three-dimensional point cloud image based on a target detection algorithm of deep learning; the goods shelf leg segmentation module is used for segmenting goods shelf legs from the area where the goods shelf is located based on a target segmentation algorithm of deep learning; the pose calculating module is used for calculating the center position of the goods shelf and the pose of the goods shelf based on the point cloud coordinates in the goods shelf legs; The point cloud completion module is used for carrying out point cloud completion on a target in the area where the goods shelf is located before the goods shelf legs are separated from the area where the goods shelf is located by a target segmentation algorithm based on deep learning; Wherein, the point cloud completion module includes: the point cloud standardization sub-module is used for converting the irregular point cloud data in the area where the goods shelf is located into standardized three-dimensional grid data; The three-dimensional convolution sub-module is used for inputting the three-dimensional grid data into a three-dimensional convolution network for processing to obtain three-dimensional grid data after convolution processing; The point cloud inverse rotor module is used for inversely converting the three-dimensional grid data after convolution processing into point cloud data; the point cloud sampling submodule is used for sampling the point cloud data subjected to the reverse conversion to obtain rough point cloud data; And the point cloud completion sub-module is used for shifting the points in the rough point cloud data, generating new points and carrying out point cloud completion on the targets in the area where the goods shelf is located.
7. A computer device, comprising: One or more processors; A storage means for storing one or more programs; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the shelf location detection method of any of claims 1-5.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the shelf position detection method according to any one of claims 1-5.

Description

Goods shelf position detection method, device, equipment and storage medium Technical Field The present invention relates to point cloud data processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting a shelf position. Background The automated guided vehicle is a transport device equipped with an automatic guiding device such as an electromagnetic device or an optical device, and capable of traveling along a predetermined path. In the course of docking an unmanned truck with a shelf, the shelf position needs to be perceived. The traditional shelf position detection method comprises the steps of using a depth camera to collect a depth image of a shelf, converting a gray value of the depth image to obtain a gray image, processing the gray image, detecting left and right column regions by adopting an edge detection algorithm based on an enhanced sobel operator, obtaining point cloud data corresponding to the left and right column regions of the shelf, extracting edge information, and calculating the spatial position and the attitude information of a center of the shelf according to a spatial geometrical relationship from the obtained edge information. Along with the increase of the complexity of the scene, the accuracy of the traditional shelf position detection method is obviously reduced. Disclosure of Invention The invention provides a method, a device, equipment and a storage medium for detecting the position of a goods shelf, which can process more complex scenes and improve the accuracy of detecting the position of the goods shelf. In a first aspect, the present invention provides a method for detecting a position of a shelf, including: acquiring a three-dimensional point cloud image shot by a depth camera; Detecting the region where the goods shelf is located from the three-dimensional point cloud image by a target detection algorithm based on deep learning; dividing shelf legs from the region where the shelf is located by a target dividing algorithm based on deep learning; and calculating the center position of the goods shelf and the gesture of the goods shelf based on the point cloud coordinates in the goods shelf legs. Optionally, a target detection algorithm based on deep learning detects an area where the shipment rack is located from the three-dimensional point cloud image, including: dividing all points in the three-dimensional point cloud image into a plurality of three-dimensional voxels with the same size; Extracting the characteristics of points in each three-dimensional voxel to obtain a characteristic matrix formed by characteristic vectors of each three-dimensional voxel; and carrying out classification detection and position regression based on the feature matrix to obtain a three-dimensional boundary box of the area where the goods shelf is located. Optionally, feature extraction is performed on points in each three-dimensional voxel to obtain a feature matrix formed by feature vectors of each three-dimensional voxel, including: Calculating the coordinate mean value of all points in the three-dimensional voxel as the centroid coordinates of the three-dimensional voxel aiming at each three-dimensional voxel; calculating the difference value between the coordinates of each point in the three-dimensional voxel and the coordinates of the mass center of the three-dimensional voxel to obtain the relative coordinates of the point and the mass center; Splicing the vector representation of the point coordinates with the vector representation of the relative coordinates to obtain a position vector of each point in the three-dimensional voxel; Inputting the position vector of the point in the three-dimensional voxel into a fully connected network, and mapping the position vector of the point in the three-dimensional voxel by the fully connected network to obtain a mapping characteristic; Inputting the mapping characteristics into a residual convolution network, and carrying out characteristic mining on the mapping characteristics by the residual convolution network to obtain characteristic vectors of the three-dimensional voxels; and combining the feature vectors of the three-dimensional voxels to form a feature matrix. Optionally, performing classification detection and position regression based on the feature matrix to obtain a three-dimensional bounding box of the area where the shelf is located, including: Inputting the feature matrix into a region generation network, processing the feature matrix by the region generation network, and carrying out classification detection and position regression on the targets in the three-dimensional point cloud image to obtain a three-dimensional boundary box of the region where the goods shelf is located. Optionally, the deep learning-based target segmentation algorithm segments shelf legs from an area where the shelf is located, including: Extracting local features of all points in the area where the