CN-122029011-A - Training platform for tactile robot

Abstract

The wearable device may include an article configured to be worn by a user and a sensor array coupled with the article. The sensor array may be configured to obtain force data, as well as other data. The system may receive information generated by a wearable device for performing a task. The information may include motion data indicative of motion of the wearable device and force data indicative of a force applied to the wearable device. The force data may be obtained from an array of sensors that generate a digital output. The system may control the robotic device to perform robotic motions based on the motion data and apply robot forces based on the force data to repeat the task. The robotic device may include an array of sensors to determine the force applied to the robotic device to control the robot force. Other aspects are described and claimed.

Inventors

• GOLDA DARIUSZ
• N. HAJI
• V. Pawatt
• D. Weissberg
• A. Bieber

Assignees

• 塔克塔系统公司

Dates

Publication Date

20260512

Application Date

20240927

Priority Date

20240207

Claims (20)

1. 1. A method, the method comprising: Receiving information generated by a wearable device for performing a task, the information including motion data indicative of motion of the wearable device and force data indicative of a force applied to the wearable device, the force data obtained from digital outputs of a sensor array of the wearable device, and Controlling a robotic device to perform a robotic motion based on the motion data and to apply a robot force based on the force data to repeat the task, the robotic device including a sensor array that generates a digital output to determine a force applied to the robotic device to control the robot force.
2. 2. The method of claim 1, wherein the sensor of the sensor array of the wearable device and the sensor of the sensor array of the robotic device perform analog-to-digital conversion to generate the digital output.
3. 3. The method of claim 1, wherein the sensor array of the wearable device and the sensor array of the robotic device each comprise a group comprising circuitry to perform analog-to-digital conversion on a group of sensors.
4. 4. The method of claim 1, wherein the wearable device is a sensing glove, and wherein the sensor array includes a portion of a sensor that generates a digital output coupled with a region of the sensing glove.
5. 5. The method of claim 1, wherein the generated information is utilized to control the robotic device in real time to effect teleoperation.
6. 6. The method of claim 1, wherein the robotic device is controlled to repeat the task a plurality of times based on the information.
7. 7. The method of claim 1, wherein the robotic device includes a plurality of actuated axes corresponding to joints of a user of the wearable device.
8. 8. The method of claim 1, the method further comprising: The task associated with the information is determined from a plurality of other tasks recorded in a library for repetition by the robotic device.
9. 9. The method of claim 1, the method further comprising: An adjustment is applied to at least one of the robotic motion or the robot force based on a difference between a state of the wearable device performing the task and another state presented to the robotic device.
10. 10. The method of claim 1, the method further comprising: An adjustment is applied to at least one of the robot motion or the robot force based on an output from a machine learning model.
11. 11. The method of claim 1, the method further comprising: Color data and depth data determined from a camera are received, the color data indicating a color of at least one of an object or a target of the object manipulated by the wearable device, and the depth data indicating a spatial orientation of the at least one of the object or the target.
12. 12. The method of claim 1, the method further comprising: the robotic device is controlled in conjunction with color data determined from a camera and depth data, the color data indicating a color of at least one of an object manipulated by the robotic device or a target of the object, and the depth data indicating a spatial orientation of at least one of the object or the target.
13. 13. The method of claim 1, wherein the sensor array of the wearable device and the sensor array of the robotic device comprise sensors that correspond to each other.
14. 14. The method of claim 1, wherein the sensor array of the wearable device and the sensor array of the robotic device are configured for multi-modal sensing.
15. 15. The method of claim 1, wherein the sensor array of the wearable device and the sensor array of the robotic device comprise microsensors.
16. 16. The method of claim 1, wherein the force applied to the wearable device and the force applied to the robotic device comprise normal forces.
17. 17. The method of claim 1, wherein the force applied to the wearable device and the force applied to the robotic device comprise shear forces.
18. 18. The method of claim 1, wherein the wearable device comprises a glove and the robotic device comprises a robotic hand.
19. 19. The method of claim 1, wherein the wearable device comprises a glove and the robotic device comprises a robotic pinch gripper.
20. 20. The method of claim 1, wherein the wearable device comprises force sensors and inertial sensors of the sensor array, the force sensors and inertial sensors having digital outputs generated based on interactions with objects in a presentation environment, Wherein the robotic device comprises a force sensor of the sensor array and an inertial sensor having digital outputs generated based on interactions with objects in a robotic environment, Wherein the digital output is used to match a force applied in the robotic environment with the force applied in the presentation environment to repeat the task in the robotic environment.

Description

Training platform for tactile robot RELATED APPLICATIONS The present application claims priority from U.S. provisional patent application No. 63/592,125 filed on 10 months 20 of 2023 and U.S. patent application No. 18/435,800 filed on 7 months 2 of 2024, each of which is incorporated herein by reference. Background Technical Field The present disclosure relates generally to robotic control, and more particularly to haptic robotic training platforms. Other aspects are also described. Background information A robotic device or robot may refer to a machine that may automatically perform one or more actions or tasks in an environment. For example, robotic devices may be configured to facilitate manufacturing, assembly, packaging, maintenance, cleaning, transportation, exploration, surgical or security protocols, and the like. The robotic device may include various mechanical components, such as robotic arms and end effectors, to interact with the surrounding environment and perform tasks. The robotic device may also include a processor or controller that executes instructions stored in the memory to configure the robotic device to perform tasks. Disclosure of Invention Implementations of the present disclosure include utilizing a wearable device and/or robotic device having a tactile sensor array configured to replicate human-scale tactile sensing, touching, grasping, and/or dexterity. In some implementations, a wearable device may include an article configured to be worn by a user and a sensor array coupled to the article. For example, the wearable device may be a sensing glove, and the article may be a glove of fabric or elastomer for wearing on the hand of the user. The sensor array may include sensors that provide multi-modal sensing (e.g., microsensing of various conditions, such as normal force, shear force, temperature, proximity, and/or images associated with an object via normal force sensors, shear force sensors, thermal sensors, proximity sensors, image sensors (e.g., RGB cameras), and/or inertial sensors, such as Inertial Measurement Units (IMUs)). In some implementations, the sensor may include a microsensor that may be sub-millimeter in at least one in-plane dimension associated with the footprint and/or may be arranged at a pitch of 3 millimeters or less (e.g., less than 3 millimeters (mm) between the footprint of the sensor). The sensor array may be configured to obtain force data (e.g., tactile sensing via normal force sensors, shear force sensors, and/or other sensing) indicative of the force applied to the wearable device, as well as other data. The sensors of the wearable device may also be configured to obtain motion data (e.g., trajectory, position, orientation, speed, or acceleration) indicative of the motion of the wearable device. Other aspects are described and claimed. The above summary does not include an exhaustive list of all aspects of the disclosure. It is intended that the present disclosure include all systems and methods that may be practiced from all suitable combinations of the various aspects outlined above, as well as those disclosed in the detailed description that follows and particularly pointed out in the claims. Such a combination may have particular advantages not specifically recited in the above summary. Drawings Several aspects disclosed herein are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements. It should be noted that references to "a" or "an" aspect in this disclosure do not necessarily refer to the same aspect, and they mean at least one. Moreover, for the sake of brevity and reducing the total number of drawings, a given drawing may be used to illustrate features of more than one aspect of the disclosure, and not all elements in the drawing may be required for a given aspect. Fig. 1 is an example of a system for controlling a robotic device with haptic sensing using a wearable device with haptic sensing. Fig. 2 is an example of an environment in which a wearable device is utilized. Fig. 3 is an example of an environment in which a robotic device is utilized. Fig. 4A is an example of a wearable device with haptic sensing, and fig. 4B is a cross-section of a portion of the wearable device. Fig. 5A is an example of a sensor array with tactile sensing, and fig. 5B is an example of a sensor generating a digital output. FIG. 6 is an example of a cross section of a sensor array with tactile sensing. Fig. 7 includes an example of haptic sensing performed by a wearable device or a robotic device. Fig. 8A is an example of a robotic device with haptic sensing, and fig. 8B is a cross-section of a portion of the robotic device. Fig. 9 is an example of a mechanical structure of the robot apparatus. Fig. 10 is an example of a control system for controlling a robotic device. FIG. 11 is an example of training a robotic device based on subtasks. Fig. 12-22 includ