CN-121290425-B - Grabbing learning method and system of intelligent robot based on small sample

CN121290425BCN 121290425 BCN121290425 BCN 121290425BCN-121290425-B

Abstract

The invention relates to the technical field of intelligent robot control, and particularly discloses a small sample-based intelligent robot grabbing and learning method and a small sample-based intelligent robot grabbing and learning system, wherein the method comprises the steps of firstly collecting a small amount of multi-mode teaching samples comprising RGB images, depth maps and touch snapshots, and constructing an initial teaching set; the method comprises the steps of pre-training an encoder by using synthesized data, fine-tuning the encoder by using real teaching samples to generate a steady prototype set, generating a pixel-level grabbing confidence level field and an uncertainty field by calculating the similarity between real-time observation and the prototype set by a grabbing evaluation module, driving virtual sample generation with physical consistency based on the uncertainty field to expand training data, carrying out quick self-adaptive fine-tuning on a grabbing model under a meta-learning framework by using an enhanced small sample set, finally solving the optimal grabbing pose by combining physical constraint, and realizing closed-loop learning by online touch probing and feedback. The invention obviously improves the grabbing success rate and the robustness of the robot in a small sample real scene.

Inventors

WAN CHEN
HUANG MIN
HU LING
ZHANG ZHIHONG
ZHOU XIA
HAN SIPING
HUANG YUHUI

Assignees

国网江西省电力有限公司抚州供电分公司

Dates

Publication Date: 20260512
Application Date: 20251113

Claims (10)

1. The grabbing and learning method of the intelligent robot based on the small sample is characterized by comprising the following steps of: S1, acquiring teaching observation of a target object, and marking and formatting the teaching observation to obtain an initial teaching set; The teaching observations comprise RGB images, depth maps, and short-time haptic or force sense snapshots; S2, performing self-supervision or contrast pre-training on the encoder by using a synthesized grabbing scene, and performing iterative fine adjustment on the pre-trained encoder by using the initial teaching set so as to map each teaching sample to a prototype embedding space to obtain a prototype set; S3, a grabbing assessment module calculates a pixel-level grabbing confidence level field and a corresponding uncertainty field based on the prototype set and the real-time RGB image-depth image to obtain a pixel-level confidence and uncertainty mapping; S4, sampling key points in an image area with the uncertainty value higher than a preset threshold based on the uncertainty field, and generating physical consistent virtual observation according to uncertainty weighting through light physical rendering or simulation so as to expand training samples, so that an enhanced small sample set is obtained; S5, taking the enhanced small sample set as a task sample, and training an outer ring and an inner ring of a grabbing model formed by the encoder and the grabbing evaluation module under a meta-learning frame to enable the grabbing model to obtain initialization capable of quickly adapting to new teaching, and obtaining the grabbing model after small sample self-adaption fine adjustment through gradient updating; S6, carrying out pixel-to-three-dimensional back projection on the pixel-level grabbing confidence level field and the depth map, solving and optimizing a candidate grabbing pose set based on shape, collision and mechanical constraint, then executing short-time touch sense and force sense trial on the pixel-level confidence, and writing success or failure results and touch sense feedback back into a new teaching sample so as to update a teaching library and be used for the next round of self-adaption, thereby obtaining a final grabbing execution result and the updated teaching library.
2. The small sample-based intelligent robot gripping learning method according to claim 1, wherein S1 comprises: N observations of the same target under multiple view angles are carried out, wherein N is more than or equal to 1 and N is less than or equal to 10, each observation simultaneously records an RGB image, a depth map corresponding to the RGB image and one or more short-time touch and force sense snapshots, and the N is used for describing object surface contact response.
3. The small sample-based intelligent robot gripping learning method according to claim 1, wherein S2 comprises: The self-supervision contrast learning based on the synthesized grabbing scene is adopted for pre-training, the loss function is contrast loss or a variation thereof, so that the encoder which is robust to geometric and material changes is obtained, and the parameter freezing ratio of the backbone of the encoder is kept to be not more than 80% during fine tuning, so that the prototype embedding space is kept universal and has domain adaptability, and a prototype set is obtained.
4. The small sample-based intelligent robot gripping learning method according to claim 1, wherein S3 comprises: Extracting pixel embedding observed in real time by the encoder; The grabbing evaluation module calculates similarity weights of pixels and all prototype vectors, obtains pixel-level confidence coefficient by adopting a weighted kernel interpolation or attention weighting mechanism, deduces pixel uncertainty based on the distribution of the similarity weights, and obtains pixel-level confidence and uncertainty mapping.
5. The small sample-based intelligent robot gripping learning method according to claim 1, wherein S4 comprises: and generating M physical consistent virtual observations in each image area with the uncertainty value higher than a preset threshold value based on the uncertainty field, and reserving depth and approximate mechanical parameters in the generation process so as to combine the virtual observations and the real teaching samples according to uncertainty weighting to form an enhanced small sample set.
6. The small sample-based intelligent robot gripping learning method according to claim 1, wherein S5 comprises: gradient updating is carried out by adopting an inner ring and outer ring training strategy based on model-independent element learning or light weight variants thereof; the outer ring optimizes the grabbing model on multiple tasks to obtain an initialization parameter which can be quickly adapted to a new task; The learning rate and the step number of the inner ring are used as adjustable super parameters, and on new teaching data provided by a target task, the initialization parameters are subjected to gradient update to obtain a small sample self-adaptive fine-tuned grabbing model; and the learning rate and the step number of the inner ring are used as adjustable super parameters so as to balance rapid adaptation and stability in a small sample scene and obtain a small sample self-adaptive fine-tuned grabbing model.
7. The small sample-based intelligent robot gripping learning method according to claim 1, wherein S6 includes: performing geometric transformation based on the depth map and camera internal parameters to obtain candidate 3D grabbing points to realize back projection from pixels to three dimensions; the solving of the grabbing pose set is locally optimized in a constraint minimization mode, and a target function of the local optimization comprises minimizing the grabbing moment, the collision penalty term and the weighted sum of grabbing uncertainty so as to output executable poses ordered according to the confidence coefficient; The method comprises the steps of executing short-time touch sense and force sense heuristics on candidate grabbing gestures according to confidence level from high to low, recording failed samples if heuristics fail, adding a teaching library according to priority to enhance negative sample learning of failed states, and setting a judging threshold of heuristics based on force change rates or absolute force values of touch sense and force sense sensors to judge grabbing success or slipping.
8. Intelligent robot snatchs learning system based on little sample, characterized by comprising: the data acquisition and processing module is used for executing the step S1 of the claim 1, acquiring multi-mode teaching observation of a target object, and performing labeling and formatting processing to obtain an initial teaching set; The model pre-training and fine tuning module is used for executing the step S2 of the claim 1, pre-training the encoder by synthesizing a grabbing scene, and performing iterative fine tuning on the pre-training encoder by utilizing the initial teaching set so as to map teaching samples to a prototype embedding space to obtain a prototype set; A capture evaluation module for performing the step S3 of claim 1, calculating a pixel-level capture confidence field and a corresponding uncertainty field based on the prototype set and real-time RGB-D observations; the sample enhancement module is used for executing the step S4 of the claim 1, sampling key points in a high uncertainty area based on the uncertainty field, and generating physical consistent virtual observation through physical rendering or simulation so as to expand training samples and obtain an enhanced small sample set; the meta-learning self-adaptation module is used for executing the step S5 of the claim 1, taking the enhanced small sample set as a task sample, performing outer ring and inner ring training on the grabbing model under a meta-learning frame, and obtaining the grabbing model after small sample self-adaptation through gradient updating; the capturing execution and feedback module is configured to execute the step S6 of claim 1, perform three-dimensional back projection on the pixel-level capturing confidence field and the depth map, solve the optimal capturing pose based on multiple constraint conditions, execute short-time haptic/force sense heuristics, and write back the result as a new teaching sample to update the teaching library.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 7 when the program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.

Description

Grabbing learning method and system of intelligent robot based on small sample Technical Field The invention belongs to the technical field of intelligent robot control, and particularly relates to a small sample-based intelligent robot grabbing and learning method and system. Background The grabbing operation of the intelligent robot is a core technology in scenes such as industrial manufacturing, home service and the like, and the core requirement is to quickly adapt to various objects in a complex real environment so as to realize grabbing with high precision and high robustness. However, the geometrical shape and texture of the target object in the real scene have large differences, and environmental factors such as illumination change, background interference and the like are complex, meanwhile, the acquisition and labeling processes of the real grabbing data have high cost and long period, and large-scale high-quality labeling samples are difficult to obtain, so that small sample learning becomes a key breakthrough direction of the robot grabbing technology in landing. Although the prior art has made some progress in the field of robotic grasping, there are still some drawbacks with small sample constraints: The traditional grabbing learning method is either based on massive real labeling data for supervision training, the data acquisition cost is extremely high and is difficult to cover diversified scenes, or the data are only pre-trained through large-scale synthesis, but an effective adaptation mechanism with the real scenes is lacking, so that significant domain deviation exists from simulation to real migration, and the generalization performance is rapidly reduced under a small amount of real teaching samples. Some methods attempt to directly fine tune the pre-trained model, but are prone to catastrophic forgetting, and it is difficult to balance model versatility with scene specificity under small sample conditions. In contrast, the application provides a small sample-based intelligent robot grabbing and learning method for solving the problems. Disclosure of Invention The invention aims to provide a small sample-based intelligent robot grabbing learning method and system, so as to solve the problems in the background technology. In order to achieve the above purpose, the present invention provides the following technical solutions: The grabbing and learning method of the intelligent robot based on the small sample comprises the following steps: s1, multi-mode small sample acquisition and teaching seed construction are used for acquiring a small amount of teaching observations of a target object, and marking and formatting the teaching observations to obtain an initial teaching set S0; The teaching observations comprise RGB images, depth maps, and short-time haptic or force sense snapshots; S2, performing self-supervision or contrast pre-training on the encoder by using a synthesized grabbing scene, and performing iterative fine adjustment on the pre-trained encoder by using the initial teaching set so as to map each teaching sample to a prototype embedding space to obtain a prototype set; S3, a grabbing assessment module calculates a pixel-level grabbing confidence level field and a corresponding uncertainty field based on the prototype set and the real-time RGB image-depth image to obtain a pixel-level confidence and uncertainty mapping; S4, sampling key points in an image area with the uncertainty value higher than a preset threshold based on the uncertainty field, and generating physical consistent virtual observation according to uncertainty weighting through light physical rendering or simulation so as to expand training samples, so that an enhanced small sample set is obtained; S5, taking the enhanced small sample set as a task sample, and training an outer ring and an inner ring of a grabbing model formed by the encoder and the grabbing evaluation module under a meta-learning frame to enable the grabbing model to obtain initialization capable of quickly adapting to new teaching, and obtaining the grabbing model after small sample self-adaption fine adjustment through gradient updating; S6, carrying out pixel-to-three-dimensional back projection on the pixel-level grabbing confidence level field and the depth map, solving and optimizing a candidate grabbing pose set based on shape, collision and mechanical constraint, then executing short-time touch sense and force sense trial on the pixel-level confidence, and writing success or failure results and touch sense feedback back into a new teaching sample so as to update a teaching library and be used for the next round of self-adaption, thereby obtaining a final grabbing execution result and the updated teaching library. Preferably, the S1 includes: n observations (N is more than or equal to 1 and N is less than or equal to 10) of the same target under multiple visual angles, and each observation records an RGB image, a depth map correspond