CN-117103245-B - Object grabbing method, object grabbing device, robot, readable storage medium and chip

CN117103245BCN 117103245 BCN117103245 BCN 117103245BCN-117103245-B

Abstract

The invention provides an object grabbing method, an object grabbing device, a robot, a readable storage medium and a chip, wherein the method comprises the steps of obtaining scene information of an environment where an object to be grabbed is located; the method comprises the steps of determining first point cloud information corresponding to an object to be grabbed according to scene information, determining a plurality of grabbing postures for grabbing the object to be grabbed according to the first point cloud information, determining a target posture in the grabbing postures, and controlling a mechanical arm to grab the object to be grabbed according to the target posture.

Inventors

TANG JIAN
ZHAO ZHEN
XU ZHIYUAN
CHE ZHENGPING

Assignees

美的集团(上海)有限公司
美的集团股份有限公司

Dates

Publication Date: 20260508
Application Date: 20230714

Claims (15)

1. An object gripping method, comprising: acquiring scene information of an environment where an object to be grabbed is located; Determining first point cloud information corresponding to the object to be grabbed according to the scene information; Filtering the first point cloud information and determining filtered second point cloud information; determining a plurality of grabbing postures for grabbing the object to be grabbed according to the second point cloud information; determining a target gesture in the plurality of grabbing gestures, and controlling a mechanical arm to grab the object to be grabbed according to the target gesture; The method comprises the steps of obtaining at least one granularity geometric filtering structure, removing abnormal point clouds in first point cloud information according to the abnormal filtering algorithm, clustering the filtered first point cloud information according to the clustering algorithm, and determining the clustered point cloud information of the largest cluster as the second point cloud information.
2. The method for capturing an object according to claim 1, wherein the acquiring scene information of an environment in which the object to be captured is located includes: and acquiring scene information of the current environment by a three-dimensional camera, wherein the scene information comprises image information and depth information of the scene, and the image information and the depth information are registered information.
3. The object gripping method according to claim 2, wherein, The determining, according to the scene information, first point cloud information corresponding to the object to be grabbed includes: and determining the first point cloud information according to the internal reference matrix of the three-dimensional camera, the image information and the depth information.
4. The object capturing method according to claim 3, wherein determining the first point cloud information from the internal reference matrix of the three-dimensional camera, the image information, and the depth information includes: acquiring a target detection network model and a segmentation network model; Identifying the image information through the target detection network model and the segmentation network model, and determining coordinate information of a first area; Determining depth information corresponding to the first region according to the coordinate information of the first region; and converting the depth information of the first area into first point cloud information according to the internal reference matrix.
5. The object grabbing method of claim 2, wherein the number of said granularity geometrical filtering structures is a plurality, and the radius distance of the point cloud in each granularity geometrical filtering structure from the surrounding neighborhood is different.
6. The object gripping method according to claim 5, characterized by further comprising: Determining the application sequence of a plurality of the granularity geometric filtering structures under the condition that the number of the granularity geometric filtering structures is a plurality of the granularity geometric filtering structures; And arranging a plurality of granularity geometric filtering structures according to the application sequence, wherein the neighborhood distance of the granularity geometric filtering structure applied in advance is smaller than that of the granularity geometric filtering structure applied in later.
7. The method for capturing an object according to claim 2, wherein the determining a plurality of capturing gestures for capturing the object to be captured according to the second point cloud information specifically includes: dividing the second point cloud information based on a geometric dividing structure to determine a target point cloud; and determining a plurality of grabbing postures corresponding to the number of the target point clouds being smaller than a stopping threshold value.
8. The object capturing method according to claim 7, wherein the segmenting the second point cloud information based on the geometric segmentation structure, determining a target point cloud, comprises: Obtaining a geometric segmentation structure and a clustering structure; separating the second point cloud information according to the geometric division structure, and determining separation point cloud information; and clustering the separated point cloud information according to the clustering structure, and determining the maximum cluster as a target point cloud.
9. The object capturing method according to claim 8, wherein the determining that the number of the target point clouds is smaller than a plurality of capturing poses corresponding to a stop threshold includes: Determining direction vectors of a plurality of target point clouds; determining a matrix corresponding to the direction vector; determining a grabbing position according to the centroid position of each target point cloud; determining a grabbing gesture corresponding to the target point cloud according to the grabbing position and the matrix; Determining the number of the target point clouds; and iteratively determining the target point cloud and the grabbing gesture until the number of the target point cloud is smaller than a stopping threshold value, stopping iteration, and determining a plurality of grabbing gestures.
10. The object grabbing method of claim 9, wherein said determining a direction vector of a plurality of said target point clouds comprises: determining direction vectors of a plurality of surface normals in the target point cloud; An average of direction vectors corresponding to a plurality of surface normals of one of the target point clouds is determined, and the average is taken as the direction vector of the target point cloud.
11. The object gripping method according to claim 9, wherein the determining a matrix corresponding to the direction vector includes: determining a corresponding Euler angle according to the direction vector; and determining a matrix according to a plurality of Euler angles.
12. A robot comprising a processor, a memory and a program or instruction stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the object gripping method according to any one of claims 1 to 11.
13. An object gripping device, comprising: The scene acquisition module is used for acquiring scene information of the environment where the object to be grabbed is located; the point cloud acquisition module is used for determining first point cloud information corresponding to the object to be grabbed according to the scene information; the filtering module is used for filtering the first point cloud information and determining filtered second point cloud information; The gesture determining module is used for determining a plurality of grabbing gestures for grabbing the object to be grabbed according to the second point cloud information; The grabbing module is used for determining a target gesture among a plurality of grabbing gestures and controlling a mechanical arm to grab the object to be grabbed according to the target gesture; The method comprises the steps of obtaining at least one granularity geometric filtering structure, removing abnormal point clouds in first point cloud information according to the abnormal filtering algorithm, clustering the filtered first point cloud information according to the clustering algorithm, and determining the clustered point cloud information of the largest cluster as the second point cloud information.
14. A readable storage medium, characterized in that the readable storage medium has stored thereon a program or instructions which, when executed by a processor, implement the steps of the object gripping method according to any of claims 1 to 11.
15. A chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being configured to execute programs or instructions for implementing the steps of the object gripping method according to any of claims 1 to 11.

Description

Object grabbing method, object grabbing device, robot, readable storage medium and chip Technical Field The present invention relates to the field of computers, and more particularly, to an object gripping method, an object gripping device, a robot, a readable storage medium, and a chip. Background In the related art, when the object is grabbed and recognized, the object is usually marked in space for grabbing and recognizing the gesture, and the simulation marking or the manual marking is adopted for a long time, so that the object cannot be analyzed and grabbed in time. Disclosure of Invention In order to solve or improve at least one of the above problems, an object of the present invention is to provide an object gripping method. Another object of the present invention is to provide an object gripping device. Another object of the present invention is to provide a robot. Another object of the present invention is to provide a readable storage medium. It is another object of the present invention to provide a chip. In order to achieve the above purpose, the first aspect of the invention provides an object grabbing method, which comprises the steps of obtaining scene information of an environment where an object to be grabbed is located, determining first point cloud information corresponding to the object to be grabbed according to the scene information, determining a plurality of grabbing postures for grabbing the object to be grabbed according to the first point cloud information, determining a target posture in the plurality of grabbing postures, and controlling a mechanical arm to grab the object to be grabbed according to the target posture. According to the object grabbing method provided by the invention, before object grabbing, the scene information of the environment where the object to be grabbed is located needs to be acquired. Such information may include object position, pose, size, position of surrounding objects, etc. And processing the acquired scene information to obtain first point cloud information corresponding to the object to be grabbed. It is understood that a point cloud is a set of three-dimensional coordinate points that can be used to represent information such as the shape and position of an object. From the first point cloud information, a plurality of possible grabbing postures may be determined, where the grabbing postures may include information such as a position and a posture of an end effector of the mechanical arm, and the grabbing postures are used for controlling the mechanical arm to grab. Finally, an optimal target gesture is selected from a plurality of possible grabbing gestures, and is used for controlling the mechanical arm to grab. In the process, the position, the gesture, the relation with surrounding objects and other factors of the object are considered, after the target gesture is determined, the mechanical arm can be moved to a proper position by controlling the motion of the end effector of the mechanical arm, parameters such as the gesture are adjusted, and finally the grabbing operation of the object to be grabbed is completed. In general, the object grabbing method utilizes the technologies of robot vision and intelligent control, combines the technologies of point cloud information processing, gesture adjustment and the like, and achieves automatic grabbing of objects to be grabbed. It is emphasized that the method is realized by directly utilizing the 3D geometric information of the object point cloud when the gesture calculation is performed, and the grabbing gesture of the marked object is not required to be performed in the 3D space, so that the marked manpower and material resources of a simulation machine are greatly saved. In addition, the technical scheme provided by the invention can also have the following additional technical characteristics: in some technical schemes, optionally, acquiring scene information of an environment where an object to be grabbed is located specifically comprises acquiring scene information of the current environment by a three-dimensional camera, wherein the scene information comprises image information and depth information of the scene, and the image information and the depth information are registered information. In the technical scheme, the three-dimensional camera can acquire the color image and the depth image of the object in the scene at the same time. The color image provides appearance information of the object, and the depth image provides distance information of the object. This information can be used for subsequent point cloud processing and object pose estimation operations. Further, the three-dimensional camera generally acquires depth information through infrared or laser technology, and can quickly and accurately acquire information such as the position and the shape of an object in a scene under different illumination conditions. In summary, by using a three-dimensional camera to obtain image