CN-122021714-A - Self-learning method of self-learning robot with body based on pseudo tag and self-learning robot system with body based on pseudo tag

CN122021714ACN 122021714 ACN122021714 ACN 122021714ACN-122021714-A

Abstract

The application relates to a self-learning method of a self-learning robot based on a pseudo tag and a self-learning robot system. The method comprises the steps of generating a state pseudo tag and a component pseudo tag related to an object to be learned based on interactive multimedia data between the robot body and the object to be learned, generating a cross-view pseudo tag related to the object to be learned based on the state pseudo tag, the component pseudo tag and the interactive multimedia data, carrying out motion simulation on the object to be learned based on the interactive multimedia data to obtain simulation multimedia data of the object to be learned, generating an enhancement pseudo tag related to the object to be learned based on the simulation multimedia data, verifying the state pseudo tag, the component pseudo tag, the cross-view pseudo tag and the enhancement pseudo tag, and training the robot body based on the verified pseudo tag. By adopting the method, the richness of the pseudo labels can be improved, and the success rate of the interaction task of the robot with the body is further improved.

Inventors

Request for anonymity
Request for anonymity
LI ZHICHEN
PAN YANG
LI JIANGANG

Assignees

卧安科技(深圳)有限公司

Dates

Publication Date: 20260512
Application Date: 20260410

Claims (11)

1. A self-learning method of a self-learning robot with a pseudo tag, the method comprising: generating a state pseudo tag and a part pseudo tag about an object to be learned based on interactive multimedia data between the robot with body and the object to be learned; generating a cross-view pseudo tag for the object to be learned based on the state pseudo tag, the component pseudo tag, and the interactive multimedia data; Performing motion simulation on the object to be learned based on the interactive multimedia data to obtain simulation multimedia data of the object to be learned, and generating an enhanced pseudo tag related to the object to be learned based on the simulation multimedia data; And verifying the state pseudo tag, the component pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag, and training the self robot based on the verified pseudo tag.
2. The method of claim 1, wherein the interactive multimedia data is acquired by an autonomous robot at a first viewing angle; The generating a cross-view pseudo tag for the object to be learned based on the state pseudo tag, the component pseudo tag, and the interactive multimedia data includes: Based on the part pseudo tag and the interactive multimedia data, carrying out three-dimensional space reprojection processing on the object to be learned to obtain a three-dimensional semantic map; Acquiring object multimedia data of the object to be learned from a second view based on the three-dimensional semantic map, wherein the second view is different from the first view; determining a component mask pseudo tag of the object to be learned under the second view angle based on the object multimedia data; Based on the component mask pseudo tag and the state pseudo tag, a cross-view pseudo tag for the object to be learned is determined.
3. The method of claim 2, wherein the determining a cross-view pseudo tag for the object to be learned based on the part mask pseudo tag and the state pseudo tag comprises: Determining the component mask pseudo tag and the state pseudo tag as cross-view pseudo tags with respect to the object to be learned when the number of the second views is one; And when the number of the second view angles is a plurality, generating view angle pseudo labels corresponding to the second view angles based on the component mask pseudo labels of each second view angle and the state pseudo labels, and determining the view angle pseudo labels corresponding to the second view angles as cross-view angle pseudo labels of the object to be learned.
4. The method according to claim 1, wherein the performing motion simulation on the object to be learned based on the interactive multimedia data to obtain simulated multimedia data of the object to be learned includes: based on the interactive multimedia data, performing joint inverse calculation on the object to be learned to obtain the motion type and corresponding motion constraint data of the object to be learned; based on the motion type and corresponding motion constraint data of the object to be learned, performing multi-state analog transformation on the object to be learned to obtain transformed multimedia data of the object to be learned under different state analog transformations; and determining the transformed multimedia data of the object to be learned under the analog transformation of different states as the simulated multimedia data of the object to be learned.
5. The method of claim 4, wherein the simulated multimedia data comprises transformed multimedia data in N state analog transforms, N being a positive integer; the generating an enhanced pseudo tag for the object to be learned based on the simulated multimedia data comprises: for state analog transformation j in the N state analog transformations, determining motion constraint data of the object to be learned under the state analog transformation j based on transformed multimedia data under the state analog transformation j; Determining motion parameters of the object to be learned under the state simulation transformation j based on the motion constraint data of the object to be learned under the state simulation transformation j and the motion type of the object to be learned; Generating a simulation pseudo tag of the object to be learned under the state simulation transformation j based on the motion parameters of the object to be learned under the state simulation transformation j and the part pseudo tag; And determining the simulation pseudo labels of the to-be-learned objects corresponding to the N states under simulation transformation as enhancement pseudo labels of the to-be-learned objects.
6. The method of claim 5, wherein generating the simulated pseudo tag for the object to be learned under the state simulation transformation j based on the motion parameters of the object to be learned under the state simulation transformation j and the component pseudo tag comprises: Determining a simulation state pseudo tag related to the object to be learned based on the motion parameters of the object to be learned under the state simulation transformation j; Acquiring a sample part mask from a memory bank of the robot body, and carrying out shielding treatment on the part pseudo tag by adopting the sample part mask to obtain a shielding part pseudo tag; And determining the simulation state pseudo tag and the shielding component pseudo tag as simulation pseudo tags of the object to be learned under the state simulation transformation j.
7. The method of claim 1, wherein the validating the status pseudo tag, the part pseudo tag, the cross-view pseudo tag, and the enhanced pseudo tag, training the self-contained robot based on validated pseudo tags, comprises: Based on preset verification logic, verifying the state pseudo tag, the part pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag respectively to obtain a pseudo tag verification result; Generating quality scores corresponding to the state pseudo tag, the part pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag respectively according to the pseudo tag verification result; Determining the verified pseudo tag based on the quality score as a qualified pseudo tag; and responding to a self-learning instruction of the self-learning robot, and performing self-learning of the self-learning robot based on the qualified pseudo tag, wherein the self-learning instruction is generated under the condition that the self-learning condition is detected to be met.
8. The method of claim 7, wherein the preset verification logic comprises motion constraint verification; The verifying, based on preset verifying logic, the state pseudo tag, the component pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag respectively to obtain a pseudo tag verification result, which includes at least one of the following: generating a pseudo tag verification result for indicating verification passing under the condition that the state pseudo tag, the component pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag are in accordance with the motion type of the object to be learned and corresponding motion constraint data; And generating a false tag verification result for indicating that verification is not passed under the condition that the motion type or corresponding motion constraint data which does not accord with the object to be learned exists in the state false tag, the component false tag, the cross-view false tag and the enhanced false tag.
9. The method of claim 7, wherein the preset verification logic comprises multi-view consistency verification; Based on preset verification logic, the state pseudo tag, the component pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag are verified respectively to obtain a pseudo tag verification result, which comprises the following steps: screening the pseudo tag with the position characteristic from the state pseudo tag, the part pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag to serve as a screening pseudo tag; determining a pseudo tag under a first visual angle from the screening pseudo tags to obtain a first visual angle pseudo tag, and performing three-dimensional projection on the first visual angle pseudo tag to obtain three-dimensional characteristics of the object to be learned under the first visual angle; determining a pseudo tag under a second visual angle from the screening pseudo tags to obtain a second visual angle pseudo tag, and performing three-dimensional projection on the second visual angle pseudo tag to obtain three-dimensional characteristics of the object to be learned under the second visual angle; and generating a pseudo tag verification result for indicating verification passing in the case that the three-dimensional feature under the first view angle is consistent with the three-dimensional feature under the second view angle.
10. The method according to any one of claims 1-9, further comprising: If a target task related to the object to be learned is received, a motion constraint model of the object to be learned is acquired, wherein the motion constraint model is learned by the robot with body based on the pseudo tag passing the verification; Determining a target state required to be reached by the object to be learned based on the target task, and determining an execution joint parameter to be executed by the robot body based on the target state, the current state of the object to be learned and the motion constraint model; And controlling the robot body to execute the executing joint parameters until the target task is completed.
11. A self-contained robotic system, the self-contained robotic system comprising: the device comprises a first generation module, a second generation module and a control module, wherein the first generation module is used for generating a state pseudo tag and a component pseudo tag related to an object to be learned based on interactive multimedia data between the robot with the body and the object to be learned; A second generation module for generating a cross-view pseudo tag for the object to be learned based on the state pseudo tag, the component pseudo tag, and the interactive multimedia data; the third generation module is used for carrying out motion simulation on the object to be learned based on the interactive multimedia data to obtain simulated multimedia data of the object to be learned, and generating an enhanced pseudo tag related to the object to be learned based on the simulated multimedia data; And the training module is used for verifying the state pseudo tag, the component pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag and training the robot based on the verified pseudo tag.

Description

Self-learning method of self-learning robot with body based on pseudo tag and self-learning robot system with body based on pseudo tag Technical Field The application relates to the technical field of self-learning robots, in particular to a self-learning method of a self-learning robot based on a pseudo tag and a self-learning robot system. Background With the development of technology in the intelligent field, the self-contained robots are increasingly widely applied and have more powerful functions, like home service robots can execute tasks according to user instructions. The robot with body needs to perform self-learning regularly in order to complete operation tasks better, and the self-learning needs to rely on a large amount of tag data. The traditional self-learning mode of the robot body has obvious defects of high manual labeling cost, low efficiency and difficulty in meeting diversified requirements, and labels predicted by the model are easy to distort, and the labels are directly used for self-learning of the robot body to cause insufficient cognition of task objects, so that the task execution of the robot body fails, and the task success rate is affected. Disclosure of Invention Based on the above, it is necessary to provide a self-learning method of a self-learning robot with a pseudo tag and a self-learning robot system with a pseudo tag, so that the richness and accuracy of the pseudo tag can be improved, and the success rate of the interaction task of the self-learning robot with a pseudo tag can be further improved. In a first aspect, the present application provides a self-learning method for a self-learning robot with a pseudo tag, including: Generating a status pseudo tag and a component pseudo tag about an object to be learned based on interactive multimedia data between the robot with body and the object to be learned; Generating a cross-view pseudo tag for the object to be learned based on the state pseudo tag, the component pseudo tag and the interactive multimedia data; Performing motion simulation on the object to be learned based on the interactive multimedia data to obtain simulated multimedia data of the object to be learned, and generating an enhanced pseudo tag related to the object to be learned based on the simulated multimedia data; And verifying the state pseudo tag, the part pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag, and training the robot body based on the verified pseudo tag. In a second aspect, the present application also provides an autonomous robot self-learning system based on a pseudo tag, including: The first generation module is used for generating a state pseudo tag and a part pseudo tag related to the object to be learned based on interactive multimedia data between the robot with the body and the object to be learned; the second generation module is used for generating a cross-view pseudo tag related to the object to be learned based on the state pseudo tag, the part pseudo tag and the interactive multimedia data; The third generation module is used for carrying out motion simulation on the object to be learned based on the interactive multimedia data to obtain simulated multimedia data of the object to be learned, and generating an enhanced pseudo tag related to the object to be learned based on the simulated multimedia data; The training module is used for verifying the state pseudo tag, the component pseudo tag, the cross-view pseudo tag and the enhanced pseudo tag, and training the robot based on the verified pseudo tag. In a third aspect, the present application also provides a computer device comprising a memory storing a computer program and a processor implementing the steps in the above-mentioned pseudo tag generation method for self-learning of an autonomous robot when the processor executes the computer program. In a fourth aspect, the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps in the above-described pseudo tag generation method for self-learning of an autonomous robot. In a fifth aspect, the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps in the above-described pseudo tag generation method for self-learning of an autonomous robot. The self-learning method of the self-learning robot based on the pseudo tag and the self-learning robot system of the self-learning robot generate a state pseudo tag and a component pseudo tag related to an object to be learned based on interactive multimedia data between the self-learning robot and the object to be learned, and generate a cross-view pseudo tag related to the object to be learned based on the state pseudo tag, the component pseudo tag and the interactive multimedia data. Further, based on the interactive multimedia data, motion simulation is carried out o