Search

CN-122023228-A - Automatic parcel sorting and pose estimating method and device for logistics conveying belt

CN122023228ACN 122023228 ACN122023228 ACN 122023228ACN-122023228-A

Abstract

The invention provides a parcel automatic classification and pose estimation method and equipment for a logistics conveying belt, which effectively inhibit false detection and repeated counting through ROI constraint and cross-frame motion prior, improve detection stability, realize real-time plane fitting and 6DoF pose estimation through parallel point cloud processing and voxel downsampling, complete frame processing under 1280×800 resolution of RGB-D images on a single 4090GPU and 14900KF CPU, meet the beat requirement of a production line, remarkably improve the sequencing quality of available targets through multidimensional confidence fusion and anomaly filtering mechanism, and preferentially output high-reliability grabbing and detecting objects.

Inventors

  • SONG XIUQIANG
  • XIANG RUI
  • PAN ZHENGYI
  • HOU DAWEI

Assignees

  • 上海微亿智造科技有限公司

Dates

Publication Date
20260512
Application Date
20251029

Claims (10)

  1. 1. The automatic parcel classifying and pose estimating method for the logistics conveying belt is characterized by comprising the following steps of: Performing ROI detection on each frame of RGB image of the package of the input logistics conveying belt to obtain a set O1 of rotating external frames of each candidate target under a pixel coordinate system; filtering the boundary based on the set O1 to obtain a set O2 of a plurality of rotating external frames in an effective operation area; Based on the set O2, performing cross-frame repeated elimination on each object based on the movement of the logistics conveyer belt to obtain an effective new target set O3 t of one frame of RGB image I at the moment t; Aiming at each target in the effective new target set O3 t , obtaining a down-sampling point cloud of each target based on the corresponding depth map; based on the down sampling point cloud of each target, obtaining a fitting plane and a corresponding 6DoF pose of each target; based on the 6DoF pose corresponding to the fitting plane of each target, multidimensional confidence calculation and weighted fusion are carried out, and mixed confidence corresponding to each target is obtained; filtering the abnormal targets based on the fitting plane of the targets and the 6DoF pose of the targets under the camera coordinate system, so as to obtain all the targets reserved after the abnormal filtering; and sequencing all the targets reserved after the anomaly filtering according to the mixed confidence level from high to low, and outputting a sequenced target information list, wherein each target information in the target information list.
  2. 2. The automatic parcel classifying and pose estimating method for a logistics conveying belt according to claim 1, wherein the ROI detection is performed on each frame of RGB image of the parcel of the input logistics conveying belt to obtain a rotating external frame of each candidate object under a pixel coordinate system, comprising: Using a trained prediction network to infer an effective working region ROI in each frame of RGB image of a package of an input logistics conveying belt to obtain a rotating external frame of each candidate target under a pixel coordinate system, and marking the rotating external frame corresponding to the ith candidate target as OBB i : Wherein class i is the class attribute (soft package, box, etc.) of the object contained in each frame of RGB image of the package of the input stream conveyer belt of the ith candidate object, Is a central point of a rotary external frame, For 4 corner points of the rotated circumscribed frame, the set of the detected rotated circumscribed frames OBB is denoted O1 = { OBB i }, And Is the two-dimensional coordinates of the center point of the rotating circumscribed frame of the candidate object, wherein, Corresponding to the lateral coordinates of the pixel coordinate system, Corresponds to the longitudinal coordinates of the pixel coordinate system.
  3. 3. The method for automatically classifying and estimating the pose of a parcel for a conveyor of claim 1, wherein the step of obtaining the set O2 of the plurality of rotating circumscribed frames within the effective operation area based on the set O1 and the boundary filtering comprises: And filtering the rotating external frames detected near the two ends of the logistics conveying belt in the collection O1, removing the rotating external frames with overlapping proportion with the effective working area ROI lower than a threshold value, and obtaining a collection O2 of a plurality of rotating external frames in the effective working area.
  4. 4. The method for automatically classifying and estimating the pose of a parcel for a logistics belt according to claim 1, wherein performing cross-frame repeated elimination based on the movement of the logistics belt for each parcel based on the set O2 to obtain an effective new target set O3 t of one frame of RGB image I at time t comprises: For one frame of RGB image I t at time t in set O2, recording that the detected target set is O2 t , and if t is the initial time, all targets in O2 t are regarded as valid new targets; For one frame of RGB image I t of t at a non-initial time, extrapolation is performed on the central position of the last frame of RGB image I t-1 in the set O2 t-1 , which is judged to be a valid new target, by using the known speed vector v of the physical conveyer belt and the adjacent frame time interval delta t to obtain a prediction set O2 t '; And performing space matching on the rotating external frame of the target in the set O2 t of the previous frame RGB image I t and the rotating external frame of the target in the prediction set O2 t ', judging that the target is repeatedly detected when the space matching of a certain target coincides and a preset threshold condition is met, removing the rotating external frame OBB of the target, and only reserving the newly-appearing rotating external frame OBB in the RGB image I t relative to the previous frame RGB image I t-1 to obtain an effective new target set O3 t corresponding to the RGB image I t .
  5. 5. The method for automatically classifying and estimating the pose of a parcel for a logistics belt according to claim 1, wherein, for each target in the set of valid new targets O3 t , based on the corresponding depth map, obtaining a down-sampling point cloud of each target comprises: For each target in the effective new target set O3 t , extracting corresponding depth in parallel on a corresponding depth map to obtain independent point clouds corresponding to each target, executing outer point filtering and downsampling operation on each independent point cloud in a preset parallel frame, and removing points with depth lower than that of the conveying belt to obtain each processed point cloud; and performing point cloud downsampling on each processed point cloud by using the voxel grids to obtain downsampled point clouds of each target.
  6. 6. The method for automatically classifying and estimating the pose of a parcel for a logistics belt according to claim 1, wherein obtaining a fitting plane and a corresponding 6DoF pose for each target based on a down-sampling point cloud for each target comprises: For each down-sampling point cloud, carrying out plane fitting by taking RANSAC plane regression as a trunk method, and supporting optional DBSCAN connectivity screening to reserve the largest continuous area so as to obtain a fitting plane of each target; after each plane fit is completed, a 6DoF pose is geometrically constructed based on the normal direction and the point cloud of each plane fit.
  7. 7. The method for automatically classifying and estimating the pose of a parcel for a logistics belt according to claim 6, wherein after each plane fitting is completed, geometrically constructing a 6DoF pose based on the normal direction and the point cloud of each plane fitting, comprising: taking a normal vector from each fitting plane, namely a plane model, normalizing the normal vector and taking the normalized normal vector as a z axis of an object coordinate system; fitting to obtain two in-plane principal axes u and v, rectangular corner points and centers through the minimum circumscribed rectangle in each fitting plane; Removing normal components from two in-plane principal axes u and normalizing to obtain an x-axis, carrying out cross multiplication on y axis ＝z axis ×x axis and normalizing to obtain a y-axis, carrying out primary re-orthogonality on the x axis ＝y axis ×z axis to ensure the three-axis orthogonality of the x-axis, the y-axis and the z-axis to obtain three orthogonal axes (x, y and z), wherein y axis represents a y-axis unit vector of an object coordinate system, z axis represents a z-axis unit vector of the object coordinate system, and x axis represents an x-axis unit vector of the object coordinate system; Three orthogonal axes (x, y and z) are respectively used as column vectors of a rotation matrix, and finally the three orthogonal axes are assembled into a complete rotation matrix of the target, which is used for describing the attitude information of the target under a camera coordinate system; And obtaining 6DoF pose corresponding to each fitting plane under a complete camera coordinate system based on the complete rotation matrix of the target, the three-dimensional angular points, the surface normal, the local coordinate axes and the sizes associated with the minimum circumscribed rectangle of the fitting plane, and optional confidence and OBB index information.
  8. 8. The method for automatically classifying and estimating the pose of the parcel for the logistics conveying belt according to claim 1, wherein the multi-dimensional confidence calculation and the weighted fusion are performed based on the 6DoF poses corresponding to the fitting planes of the objects, so as to obtain the mixed confidence corresponding to each object, and the method comprises the following steps: Based on 6DoF pose of each target under the complete camera coordinate system corresponding to the fitting plane, calculating each component such as depth confidence, area confidence, shielding confidence and angle confidence of each target, and normalizing each component to a [0,1] interval respectively; Based on the fitting plane of the target and the 6DoF pose under the camera coordinate system, filtering the abnormal target to obtain all the targets reserved after the abnormal filtering, including: Based on the fitting plane of the target and the 6DoF pose of the target under the camera coordinate system, an anomaly discrimination rule is set, the fitting result is filtered, the unreliable target is removed, and all the targets reserved after anomaly filtering are obtained.
  9. 9. A computer-readable storage medium having stored thereon computer-executable instructions, wherein execution of the computer-executable instructions by a processor causes the processor to perform the method of any one of claims 1 to 8.
  10. 10. A calculator device, comprising: Processor, and A memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of any one of claims 1 to 8.

Description

Automatic parcel sorting and pose estimating method and device for logistics conveying belt Technical Field The invention relates to a method and equipment for automatically classifying and estimating the pose of a parcel facing a logistics conveying belt. Background In the traditional industrial logistics conveyor belt scenario, the estimation of the material thereon has the following drawbacks: 1. the precision is poor, namely the high-speed movement of the conveyer belt and the rapid relative displacement of the package make the real-time property and the precision difficult to be compatible, and the omission factor, the late detection and the positioning lag are caused. 2. The robustness is insufficient, the package appearance and the pose are various, the static template or the single-view 2D feature is insensitive to the pose, and category confusion and pose estimation deviation are easy to cause. 3. And the separation and tracking are difficult, the boundary segmentation is unclear due to stacking and partial shielding, the ID is frequently switched, and the classification statistics and the track consistency are destroyed. 4. Repeated detection of long time sequence, namely that the package can be repeatedly detected in the long time sequence, and the cognitive abnormality of the downstream task is caused. 5. The reusability and standardization are insufficient, the interfaces are not uniform, the calibration and parameter configuration are complex, and the quick on-line and maintenance of the cross-production line/machine position are limited. Disclosure of Invention The invention aims to provide a method and equipment for automatically classifying packages and estimating the pose of the packages facing a logistics conveying belt. In order to solve the above problems, the present invention provides a method for automatically classifying and estimating the pose of a parcel for a logistics conveyor belt, comprising: Performing ROI detection on each frame of RGB image of the package of the input logistics conveying belt to obtain a set O1 of rotating external frames of each candidate target under a pixel coordinate system; filtering the boundary based on the set O1 to obtain a set O2 of a plurality of rotating external frames in an effective operation area; Based on the set O2, performing cross-frame repeated elimination on each object based on the movement of the logistics conveyer belt to obtain an effective new target set O3 t of one frame of RGB image I at the moment t; Aiming at each target in the effective new target set O3 t, obtaining a down-sampling point cloud of each target based on the corresponding depth map; based on the down sampling point cloud of each target, obtaining a fitting plane and a corresponding 6DoF pose of each target; based on the 6DoF pose corresponding to the fitting plane of each target, multidimensional confidence calculation and weighted fusion are carried out, and mixed confidence corresponding to each target is obtained; filtering the abnormal targets based on the fitting plane of the targets and the 6DoF pose of the targets under the camera coordinate system, so as to obtain all the targets reserved after the abnormal filtering; and sequencing all the targets reserved after the anomaly filtering according to the mixed confidence level from high to low, and outputting a sequenced target information list, wherein each target information in the target information list. Further, in the above method, ROI detection is performed on each frame of RGB image of the package of the input stream conveyor belt to obtain a rotating external frame of each candidate target in a pixel coordinate system, including: Using a trained prediction network to infer an effective working region ROI in each frame of RGB image of a package of an input logistics conveying belt to obtain a rotating external frame of each candidate target under a pixel coordinate system, and marking the rotating external frame corresponding to the ith candidate target as OBB i: Wherein class i is the class attribute (soft package, box, etc.) of the object contained in each frame of RGB image of the package of the input stream conveyer belt of the ith candidate object, Is a central point of a rotary external frame,For 4 corner points of the rotated circumscribed frame, the set of the detected rotated circumscribed frames OBB is denoted O1 = { OBB i }, AndIs the two-dimensional coordinates of the center point of the rotating circumscribed frame of the candidate object, wherein,Corresponding to the lateral coordinates of the pixel coordinate system,Corresponds to the longitudinal coordinates of the pixel coordinate system. Further, in the above method, filtering the boundary based on the set O1 to obtain a set O2 of a plurality of rotating circumscribed frames within the effective operation area, including: And filtering the rotating external frames detected near the two ends of the logistics conveying belt in the co