CN-116188573-B - Object gesture recognition method, object gesture recognition device, computer equipment, storage medium and product

CN116188573BCN 116188573 BCN116188573 BCN 116188573BCN-116188573-B

Abstract

The application relates to an object gesture recognition method, a device, a computer device, a storage medium and a product, which are characterized in that a scene image is acquired, a sub-image corresponding to a target object is acquired from the scene image, at least one visible surface of the target object is acquired based on sub-image recognition, and detecting a plurality of characteristic points in the subgraph, carrying out matching processing on at least one visible surface and the plurality of characteristic points to obtain a target data set of a target object, and finally obtaining the object posture of the target object based on the target data set, so that the method is suitable for object posture identification of various objects and can accurately obtain the object posture.

Inventors

WU YIDONG
SONG LINGXIAO
Han Mingshuo

Assignees

上海非夕机器人科技有限公司
非夕科技有限公司

Dates

Publication Date: 20260505
Application Date: 20230202

Claims (10)

1. An object gesture recognition method, the method comprising: Acquiring a scene image, and acquiring a subgraph corresponding to a target object from the scene image, wherein the target object comprises a plug; the method comprises the steps of identifying one or two visible faces of a target object based on the subgraph, detecting all visible angular points in the subgraph, numbering all the visible angular points according to a preset numbering rule, and outputting serial numbers corresponding to the identified visible angular points, wherein the visible angular points are characteristic points; Matching the one or two visible surfaces and the plurality of characteristic points to obtain a target data set of the target object, wherein the target data set comprises a target visible surface and matched target characteristic points; and obtaining the object posture of the target object based on the target data set.
2. The method according to claim 1, wherein the scene image includes a plurality of objects, and the obtaining the sub-graph corresponding to the target object from the scene image includes: detecting all objects in the scene image, and acquiring object labels of each object in all objects; obtaining a target object label corresponding to a target object; matching the target object tag with a plurality of object tags, and determining the area where the target object is located; And processing the region where the target object is located to obtain a sub-graph corresponding to the target object.
3. The method according to claim 1, wherein said matching the at least one visible surface and the plurality of feature points to obtain a target dataset of the target object comprises: Matching the at least one visible surface and the plurality of feature points to obtain at least one candidate data set, wherein the candidate data set comprises one visible surface and a key point group corresponding to the visible surface, and the key point group comprises a preset number of feature points; And verifying each key point group in each group of candidate data sets, and screening a target data set from the candidate data sets based on a verification result.
4. A method according to claim 3, wherein the verifying each of the keypoints in each set of candidate data sets, and screening the target data set from the candidate data sets based on the verification result, comprises: Checking the area of a polygon formed by the key point groups aiming at the key point groups in any candidate data set, and calculating the parallelogram degree according to the key point groups if the area is larger than an area threshold; And outputting the candidate data set corresponding to the key point group with the parallelogram degree meeting the preset condition as a target data set.
5. The method of claim 4, wherein said calculating a parallelogram degree from said set of keypoints comprises: taking any characteristic point in the key point group as a target point in sequence; Performing parallelogram fitting on the rest characteristic points in the key point group aiming at each selected target point to obtain a predicted point, and calculating the distance between the target point and the predicted point; and taking the maximum value of the calculated distances as the parallelogram degree.
6. The method according to claim 1, wherein the target feature points are two-dimensional feature points, the deriving the object pose of the target object based on the target dataset comprising: Selecting a target prior three-dimensional characteristic point corresponding to the target visible surface from a plurality of prior three-dimensional characteristic points according to the target visible surface in the target data set; and performing dimension conversion based on the target prior three-dimensional feature points and the target feature points to obtain the object posture of the target object.
7. The method of claim 6, wherein said performing a dimension transformation based on said target prior three-dimensional feature points and said target feature points to obtain an object pose of said target object comprises: Performing dimension conversion based on the target prior three-dimensional feature points and the target feature points to obtain candidate postures of the target object; calculating position information of a central point of a quadrangle surrounded by the target characteristic points; and updating the coordinate values of the transverse axis and the longitudinal axis in the candidate gestures based on the position information of the center point to obtain the object gesture of the target object.
8. An object gesture recognition apparatus, the apparatus comprising: The device comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a scene image and acquiring a sub-image corresponding to a target object from the scene image, wherein the target object comprises a plug; The detection module is used for obtaining one or two visible surfaces of the target object based on the subgraph identification and detecting all visible corner points in the subgraph; numbering all the visible corner points according to a preset numbering rule, and outputting serial numbers corresponding to each identified visible corner point, wherein the visible corner points are characteristic points; The matching module is used for carrying out matching processing on the one or two visible surfaces and the plurality of characteristic points to obtain a target data set of the target object, wherein the target data set comprises a target visible surface and matched target characteristic points; and the gesture module is used for obtaining the object gesture of the target object based on the target data set.
9. The apparatus of claim 8, wherein the acquisition module comprises a detection unit, an acquisition unit, and a processing unit, The detection unit is used for detecting all objects in the scene image and acquiring object labels of each object in all objects; The acquisition unit is used for acquiring a target object label corresponding to the target object; The processing unit is used for matching the target object label with a plurality of object labels to determine the area where the target object is located, and processing the area where the target object is located to obtain a subgraph corresponding to the target object.
10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.

Description

Object gesture recognition method, object gesture recognition device, computer equipment, storage medium and product Technical Field The present application relates to the field of industrial vision technology, and in particular, to a method, an apparatus, a computer device, a storage medium, and a product for recognizing an object gesture. Background In the industrial field, in order to save manpower and reduce cost, machines are increasingly used for replacing manpower, for example, mechanical arms are used for replacing manpower to pull out and insert plugs. Since the plug needs to be inserted in a fixed direction, recognition of the posture of the plug is important. In the traditional technology, the 3D camera is used for carrying out gesture recognition on the plug, and the 3D camera can shoot a stereoscopic plug image, so that the plug gesture in the plug image is recognized. However, due to the limitation of the 3D point cloud precision of the camera, too small or too complex plugs cannot be modeled, so that errors occur in gesture recognition. Disclosure of Invention In view of the foregoing, it is desirable to provide an object posture recognition method, apparatus, computer device, storage medium, and product capable of accurately recognizing the posture of an object. In a first aspect, the present application provides a method for recognizing a gesture of an object, the method comprising: acquiring a scene image, and acquiring a subgraph corresponding to a target object from the scene image; Obtaining at least one visible surface of the target object based on the subgraph identification, and detecting a plurality of characteristic points in the subgraph; Performing matching processing on at least one visible surface and a plurality of characteristic points to obtain a target data set of a target object, wherein the target data set comprises the target visible surface and the matched target characteristic points; an object pose of the target object is obtained based on the target dataset. In one embodiment, the scene image includes a plurality of objects, and the step of obtaining a sub-image corresponding to the target object from the scene image includes: Detecting all objects in the scene image, and acquiring object labels of each object in all objects; Acquiring a target object label corresponding to a target object; Matching the target object tag with a plurality of object tags to determine the area where the target object is located; And processing the region where the target object is located to obtain a subgraph corresponding to the target object. In one embodiment, the step of detecting a plurality of feature points in the subgraph comprises: detecting all visible corner points in the subgraph; Numbering all the visible corner points according to a preset numbering rule, and outputting the serial numbers corresponding to the identified visible corner points, wherein the visible corner points are characteristic points. In one embodiment, the step of matching the at least one visible surface with the plurality of feature points to obtain the target data set of the target object includes: Matching at least one visible surface and a plurality of characteristic points to obtain at least one group of candidate data sets, wherein each candidate data set comprises one visible surface and a key point group corresponding to the visible surface, and each key point group comprises a preset number of characteristic points; And respectively checking the key points in each group of candidate data sets, and screening out the target data set from the candidate data sets based on the checking result. In one embodiment, the step of verifying each of the keypoints in each of the candidate data sets and screening the target data set from the candidate data sets based on the verification result includes: Checking the area of the polygon formed by the key point groups aiming at the key point group in any candidate data set, and calculating the parallelogram degree according to the key point groups if the area is larger than an area threshold value; And outputting the candidate data set corresponding to the key point group with the parallelogram degree meeting the preset condition as a target data set. In one embodiment, the step of calculating the parallelogram degree from the key point group includes: Taking any characteristic point in the key point group as a target point in sequence; aiming at each selected target point, carrying out parallelogram fitting on the rest characteristic points in the key point group to obtain a predicted point, and calculating the distance between the target point and the predicted point; and taking the maximum value of the calculated distances as the parallelogram degree. In one embodiment, the target feature point is a two-dimensional feature point, and the step of obtaining the object pose of the target object based on the target data set includes: Selecting a target prior thr