CN-122029015-A - Method, apparatus, device and medium for determining actions of a robotic device

CN122029015ACN 122029015 ACN122029015 ACN 122029015ACN-122029015-A

Abstract

Provided are an action method, apparatus, device and medium for determining a robot device. In one method, a user task is received, the user task instructing a robotic device to acquire a target item of a target type. A set of image sequences of an environmental space in which the robotic device is located is acquired, the image sequences in the set of image sequences comprising a first plurality of images of the robotic device at a first plurality of points in time, respectively. In response to determining that the set of image sequences includes the target object, a sequence of states of the robotic device is acquired. Determining a second plurality of actions of the robotic device at a second plurality of points in time, respectively, based on the set of image sequences and the state sequences, using the action model, the second plurality of actions being for controlling the robotic device to acquire the target object. The historical state of the robotic device and the collected historical images may be utilized to determine actions to be performed in the future, thereby acquiring the target item in a more accurate and efficient manner.

Inventors

LI YIFENG
ZHENG ZILIN
KONG TAO

Assignees

北京有竹居网络技术有限公司

Dates

Publication Date: 20260512
Application Date: 20240910

Claims (14)

An action method for determining a robotic device, comprising: Receiving a user task, wherein the user task instructs the robot equipment to acquire a target object of a target type; Acquiring a group of image sequences of an environment space in which the robot equipment is located, wherein the image sequences in the group of image sequences comprise a first plurality of images of the robot equipment at a first plurality of time points respectively; Acquiring a sequence of states of the robotic device in response to determining that the set of image sequences includes the target object, the sequence of states including a first plurality of states of the robotic device at the first plurality of points in time, respectively, and Determining, using an action model, a second plurality of actions of the robotic device at a second plurality of points in time, respectively, based on the set of image sequences and the state sequences, the second plurality of points in time being subsequent to the first plurality of points in time, and the second plurality of actions being for controlling the robotic device to acquire the target item.
The method of claim 1, further comprising: Determining a second plurality of states of the robotic device at the second plurality of points in time, respectively, based on the second plurality of actions, and Performing a smoothing process for the second plurality of states to update a second plurality of actions of the robotic device at the second plurality of points in time.
The method of claim 1, wherein the user task further indicates a rule for obtaining the target item, and determining the second plurality of actions further comprises determining the second plurality of actions based on the rule using the action model.
The method of claim 3, wherein the rules include at least any one of a number of the target items, a requirement for acquisition of the target items, and a constraint for acquisition of the target items.
The method of claim 1, wherein the action model is determined based on: Acquiring a first group of reference image sequences of a reference environment space in which a reference robot device is located, wherein the reference image sequences in the first group of reference image sequences comprise a first plurality of reference images of the reference robot device at a first plurality of reference time points respectively, and the reference robot device acquires a reference target object; acquiring a first reference state sequence of the reference robot device, wherein the first reference state sequence comprises a first plurality of reference states of the reference robot device at the first plurality of reference time points respectively; acquiring a second sequence of reference states of the reference robotic device, the second sequence of reference states comprising a second plurality of reference states of the reference robotic device at a second plurality of reference time points, respectively, the second plurality of reference time points being subsequent to the first plurality of reference time points, and Updating the motion model using the first set of reference image sequences, the first reference state sequence, and the second reference state sequence.
The method of claim 5, wherein updating the action model further comprises: acquiring a second set of reference image sequences of the reference environment space, the reference image sequences of the second set of reference image sequences comprising a second plurality of reference images of the reference robotic device at the second plurality of reference time points, respectively, and Updating the motion model using the second plurality of reference images.
The method of claim 5, wherein updating the action model further comprises: Determining a reference rule for acquiring the reference target object, and And updating the action model by using the reference rule.
The method of claim 5, wherein acquiring the first set of reference image sequences comprises: Controlling the reference robotic device with a reference motion trajectory at the first plurality of reference points in time for a reference image sequence of the set of reference image sequences, and And respectively acquiring a first plurality of reference images of the reference robot equipment at the first plurality of reference time points.
The method of claim 1, further comprising moving the robotic device in response to determining that the images in the set of image sequences do not include the target item.
The method of claim 1, wherein the first set of image sequences comprises a first image sequence acquired by a first acquisition unit disposed at a first location and a second image sequence acquired by a second acquisition unit disposed at a second location.
An action apparatus for determining a robotic device, comprising: The receiving module is configured to receive a user task, wherein the user task indicates the robot equipment to acquire a target object of a target type; an image acquisition module configured to acquire a set of image sequences of an environmental space in which the robot device is located, the image sequences in the set of image sequences including a first plurality of images of the robot device at a first plurality of points in time, respectively; A state acquisition module configured to acquire a state sequence of the robotic device in response to determining that the set of image sequences includes the target object, the state sequence including a first plurality of states of the robotic device at the first plurality of points in time, respectively, and A determining module configured to determine, based on the set of image sequences and the state sequence, a second plurality of actions of the robotic device at a second plurality of points in time, respectively, the second plurality of points in time being subsequent to the first plurality of points in time, and the second plurality of actions being for controlling the robotic device to acquire the target item, using an action model.
An electronic device, comprising: at least one processing unit, and At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which when executed by the at least one processing unit, cause the electronic device to perform the method of any one of claims 1 to 10.
A computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to implement the method of any of claims 1 to 10.
A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 10.

Description

Method, apparatus, device and medium for determining actions of a robotic device Technical Field Example implementations of the present disclosure relate generally to the field of robots and, more particularly, relate to methods, apparatuses, devices, and computer-readable storage media for determining actions of a robotic device using a robot. Background Robotics have been rapidly developed and have been widely used in a variety of technical fields. A variety of specialized robotic devices have been developed, for example, in an industrial environment, robots may be used to perform a variety of tasks such as processing, grasping, sorting, packaging, and the like. As another example, in a home environment, a floor sweeping robot, a glass wiping robot, and the like have been developed. However, the accuracy with which the robotic device performs various tasks is not satisfactory, and thus it is desirable to improve the accuracy of the robotic device to perform a specified task in a more accurate manner. Disclosure of Invention In a first aspect of the present disclosure, a method for determining an action of a robotic device is provided. In the method, a user task is received, the user task instructing the robotic device to acquire a target item of a target type. A set of image sequences of an environmental space in which the robotic device is located is acquired, the image sequences in the set of image sequences comprising a first plurality of images of the robotic device at a first plurality of points in time, respectively. In response to determining that the set of image sequences includes the target object, a sequence of states of the robotic device is acquired, the sequence of states including a first plurality of states of the robotic device at a first plurality of points in time, respectively. Determining, using the motion model, a second plurality of motions of the robotic device at a second plurality of points in time, respectively, based on the set of image sequences and the state sequences, the second plurality of points in time being subsequent to the first plurality of points in time, and the second plurality of motions being for controlling the robotic device to acquire the target object. In a second aspect of the present disclosure, an action device for determining a robotic device is provided. The device comprises a receiving module, an image acquisition module, a state acquisition module and a determining module, wherein the receiving module is used for receiving a user task, the user task indicates the robot device to acquire a target object of a target type, the image acquisition module is used for acquiring a group of image sequences of an environment space where the robot device is located, the image sequences in the group of image sequences comprise a first plurality of images of the robot device at a first plurality of time points respectively, the state acquisition module is used for responding to determining that the group of image sequences comprise the target object, the state sequences of the robot device comprise a first plurality of states of the robot device at the first plurality of time points respectively, and the determining module is used for determining a second plurality of actions of the robot device at a second plurality of time points respectively based on the group of image sequences and the state sequences, the second plurality of actions are used for controlling the robot device to acquire the target object. In a third aspect of the present disclosure, an electronic device is provided. The electronic device comprises at least one processing unit, and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which when executed by the at least one processing unit, cause the electronic device to perform a method according to the first aspect of the disclosure. In a fourth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to implement a method according to the first aspect of the present disclosure. In a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program, wherein the computer program when executed by a processor implements the method according to the first aspect of the present disclosure. It should be understood that what is described in this section of this disclosure is not intended to limit key features or essential features of the implementations of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following description. Drawings The above and other features, advantages, and aspects of various implementations of the present disclosure will become more apparent hereinafter with reference to the