Search

CN-121973193-A - Object sorting method, device, equipment, medium and product

CN121973193ACN 121973193 ACN121973193 ACN 121973193ACN-121973193-A

Abstract

The invention discloses an object sorting method, an object sorting device, an object sorting equipment, an object sorting medium and an object sorting product. The method comprises the steps of obtaining 3D point cloud data of a scene to be sorted, inputting the 3D point cloud data into a pre-trained double-branch point cloud instance segmentation model to conduct instance segmentation to obtain clustering data of at least one object instance, inputting the clustering data of each instance into a scoring sub-model of the segmentation model to obtain instance score of each instance, selecting an instance with the instance score higher than a preset threshold and the highest instance score as a target instance, conducting gesture estimation on the target instance, determining grabbing points of the target instance, and controlling a robot to execute grabbing operation on the target instance. The embodiment of the invention can realize high-precision autonomous grabbing of the object.

Inventors

  • YAN YUYAO
  • ZHANG YIMING
  • ZHAO WEIGUANG
  • YANG XI
  • WANG QIUFENG

Assignees

  • 西交利物浦大学

Dates

Publication Date
20260505
Application Date
20260121

Claims (10)

  1. 1. A method of sorting objects, the method comprising: inputting the 3D point cloud data into a pre-trained double-branch point cloud instance segmentation model for instance segmentation to obtain cluster data of at least one object instance, wherein the segmentation model extracts semantic features and center offset features of the 3D point cloud data through a multi-scale feature fusion feature extraction neural network, and completes instance clustering based on density estimation and a neighbor voting mechanism; inputting the clustering data of each instance to a scoring sub-model of the segmentation model to obtain an instance score of each instance, and selecting an instance with an instance score higher than a preset threshold and highest instance score as a target instance; And carrying out gesture estimation on the target instance, determining a grabbing point of the target instance, and controlling a robot to execute grabbing operation on the target instance.
  2. 2. The method of claim 1, wherein the clustering of instances based on the density estimate and the neighbor voting mechanism comprises: For the original point of the 3D point cloud data, determining the reconstruction position of the point according to the original position of the point and the center offset characteristic, determining the density estimation of the point based on the number of the points in the neighborhood of the reconstruction position, and dividing the point into a high-density point and a low-density point according to the density estimation result and a preset density threshold; generating an initial cluster for the high-density points based on density clustering; distributing the low-density points to adjacent initial clusters through a neighbor voting mechanism; The neighbor voting mechanism comprises the steps of determining high-density points which are successfully allocated to initial clusters in the spatial neighborhood of the low-density points as adjacent auxiliary points of the low-density points, counting the cluster categories to which the adjacent auxiliary points belong, and allocating the low-density points to the cluster category with the most adjacent auxiliary points.
  3. 3. The method of claim 1, wherein the segmentation model extracts semantic features and center-offset features of the 3D point cloud data through a feature extraction neural network of multi-scale feature fusion, comprising: extracting semantic features and center offset features through the feature extraction neural network of the multi-scale feature fusion, and respectively carrying out semantic prediction and offset prediction through two parallel branches; the semantic features are used for representing the object category to which each point belongs, and the center offset features are used for representing the spatial displacement of each point pointing to the center of the object instance to which each point belongs, so that the points belonging to the same instance are gathered to the common center in space, and a basis is provided for the subsequent instance segmentation.
  4. 4. The method of claim 1, wherein said performing pose estimation on said target instance, determining a grabbing point of said target instance, comprises: controlling a manipulator of a robot to move above a target instance, and determining a contour point set of the target instance under a plane coordinate system opposite to the manipulator; Selecting two feature points capable of representing the main direction of the target example from the contour point set to form a main direction feature edge; Obtaining a characteristic width w according to the projection of the main direction characteristic edge on a transverse axis x of a plane coordinate system, obtaining a characteristic height h according to the projection of the main direction characteristic edge on a longitudinal axis y of the plane coordinate system, and determining an included angle theta between the main direction characteristic edge and the transverse axis x; determining long sides and short sides of the target example according to tangent values of the ratio of the characteristic height h to the characteristic width w, wherein the surface where the long sides are positioned is used as a grabbing surface of the target example; and determining the position of the grabbing point and the approaching gesture of the manipulator grabbing clamp according to the direction of the grabbing surface and the included angle theta.
  5. 5. The method of claim 1, wherein prior to said performing pose estimation on said target instance, determining a gripping point of said target instance, controlling a robot to perform a gripping operation on said target instance further comprises: controlling a manipulator of the robot to move at least three different poses, scanning a calibration plate in a working area through a point cloud data acquisition device under each pose, acquiring point cloud data of the calibration plate under an acquisition device coordinate system, and synchronously recording pose data of the current manipulator; determining a conversion matrix from the acquisition equipment coordinate system to the manipulator coordinate system through the corresponding relation between the plurality of groups of point cloud data and the manipulator pose data; and according to the conversion matrix, converting the point cloud data of the target instance from an acquisition equipment coordinate system to a manipulator coordinate system.
  6. 6. The method of claim 1, further comprising, after controlling the robot to perform a grabbing operation on the target instance: And after the target instance is removed, re-acquiring 3D point cloud data of the scene to be sorted, and sequentially grabbing the rest object instances in the scene until all the object instances with the instance scores higher than a preset threshold value are grabbed.
  7. 7. An object sorting apparatus, the apparatus comprising: the point cloud data acquisition module is used for acquiring 3D point cloud data of a scene to be sorted; The system comprises an instance segmentation module, a clustering module, a density estimation and neighbor voting mechanism, a clustering module and a clustering module, wherein the instance segmentation module is used for inputting the 3D point cloud data into a pre-trained double-branch point cloud instance segmentation model to conduct instance segmentation to obtain cluster data of at least one object instance, and the segmentation model extracts semantic features and center offset features of the 3D point cloud data through a feature extraction neural network fused by multi-scale features and completes instance clustering based on density estimation and neighbor voting mechanism; the target instance determining module is used for inputting the clustering data of each instance into the scoring sub-model of the segmentation model to obtain the instance score of each instance, and selecting the instance with the instance score higher than a preset threshold value and the highest instance score as the target instance; and the example grabbing module is used for carrying out gesture estimation on the target example, determining grabbing points of the target example and controlling the robot to execute grabbing operation on the target example.
  8. 8. An electronic device, the electronic device comprising: At least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the object sorting method of any one of claims 1-6.
  9. 9. A computer readable storage medium storing computer instructions for causing a processor to perform the method of sorting objects of any one of claims 1-6 when executed.
  10. 10. A computer program product comprising a computer program which, when executed by a processor, implements the object sorting method according to any of claims 1-6.

Description

Object sorting method, device, equipment, medium and product Technical Field The invention relates to the technical field of artificial intelligence, in particular to an object sorting method, an object sorting device, an object sorting equipment, an object sorting medium and an object sorting product. Background In industrial applications, the robot autonomous grasping technology is a key to improving production efficiency and reducing labor cost. The prior art has limited ability to identify and match objects when dealing with non-textured or less textured industrial objects, or has low gripping success due to lack of efficient three-dimensional spatial recognition capabilities for complex or stacked objects. The robot has the advantages that the recognition and grabbing capacity of the robot on complex objects in an industrial environment is improved, the object sorting is achieved through high-precision autonomous grabbing, and the robot has important significance in improving the industrial automation level. Disclosure of Invention The invention provides an object sorting method, an object sorting device, an object sorting equipment, an object sorting medium and an object sorting product, which are used for solving the problem that the grabbing success rate is not high due to the lack of effective three-dimensional space recognition capability when the capability of processing non-textured or less-textured industrial objects is limited or complex or stacked objects in the prior art. According to an aspect of the present invention, there is provided an object sorting method including: acquiring 3D point cloud data of a scene to be sorted; Inputting the 3D point cloud data into a pre-trained double-branch point cloud instance segmentation model for instance segmentation to obtain clustering data of at least one object instance, wherein the segmentation model extracts semantic features and center offset features of the 3D point cloud data through a multi-scale feature fused feature extraction neural network, and completes instance clustering based on density estimation and a neighbor voting mechanism; inputting the clustering data of each instance to a scoring sub-model of the segmentation model to obtain an instance score of each instance, and selecting an instance with an instance score higher than a preset threshold and highest instance score as a target instance; And carrying out gesture estimation on the target instance, determining a grabbing point of the target instance, and controlling a robot to execute grabbing operation on the target instance. According to another aspect of the present invention, there is provided an object sorting apparatus including: the point cloud data acquisition module is used for acquiring 3D point cloud data of a scene to be sorted; The system comprises an instance segmentation module, a clustering module, a density estimation and neighbor voting mechanism, a clustering module and a clustering module, wherein the instance segmentation module is used for inputting the 3D point cloud data into a pre-trained double-branch point cloud instance segmentation model to conduct instance segmentation to obtain cluster data of at least one object instance, and the segmentation model extracts semantic features and center offset features of the 3D point cloud data through a feature extraction neural network fused by multi-scale features and completes instance clustering based on density estimation and neighbor voting mechanism; the target instance determining module is used for inputting the clustering data of each instance into the scoring sub-model of the segmentation model to obtain the instance score of each instance, and selecting the instance with the instance score higher than a preset threshold value and the highest instance score as the target instance; and the example grabbing module is used for carrying out gesture estimation on the target example, determining grabbing points of the target example and controlling the robot to execute grabbing operation on the target example. According to another aspect of the invention, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the object sorting method according to any of the embodiments of the invention. According to another aspect of the present invention, there is provided an electronic apparatus including: The object sorting system comprises at least one processor, and a memory communicatively connected with the at least one processor, wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the object sorting method according to any embodiment of the invention. According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a pr