WO-2026094178-A1 - ROBOT CONTROL DEVICE, ROBOT CONTROL METHOD, AND PROGRAM

Abstract

The present invention relates to a robot control device for controlling a robot, the robot control device comprising: an arrival point estimator that estimates a target position of the robot on the basis of observation information relating to the state of work performed by the robot; and a plurality of constraint expressions estimator that estimates a plurality of constraints to be adhered to during the work by the robot and the importance of each of the plurality of constraints on the basis of the observation information and verbal instructions for the robot.

Inventors

• MIYAHARA, MASATO
• SESHIMO, HITOSHI
• MATSUMURA, Narimune
• ISHIHARA, TATSUYA
• TAKAGI, MOTOHIRO
• TAKAHASHI, KOUHEI
• KANADA, Taichi

Assignees

• ＮＴＴ株式会社

Dates

Publication Date

20260507

Application Date

20241030

Claims (4)

1. A robot control device for controlling a robot, A destination estimator that estimates the target position of the robot based on observational information regarding the work status of the robot, A robot control device comprising a multiple constraint representation estimator that estimates a plurality of constraints to be observed when the robot performs work, and the importance of each of the plurality of constraints, based on the aforementioned observation information and linguistic instructions to the robot.
2. The robot control device according to claim 1, further comprising a trajectory generator that generates a robot trajectory that adheres to each of the multiple constraints, based on the set of each of the multiple constraints and their importance, and the target position.
3. A robot control method performed by a robot control device for controlling a robot, The processor of the robot control device The steps include: estimating the target position of the robot based on observational information regarding the robot's work status; A step of estimating, based on the observation information and the verbal instructions given to the robot, a plurality of constraints that the robot must adhere to during its work, and the importance of each of the plurality of constraints, A robot control method that performs the step of generating a robot trajectory that adheres to each of the multiple constraints, based on a set of each of the multiple constraints and their importance, and the target position.
4. A function to estimate the target position of the robot based on observational information regarding the robot's work status. A function that estimates multiple constraints that the robot must adhere to during its work, and the importance of each of these constraints, based on the aforementioned observational information and linguistic instructions given to the robot. A program for enabling a processor to perform a function that generates a robot trajectory that adheres to each of the aforementioned constraints, based on the set of each of the aforementioned constraints and their importance, and the target position.

Description

Robot control device, robot control method, and program This invention relates to a robot control device, a robot control method, and a program for controlling a robot while considering safety within its working range. In recent years, several robot control technologies have emerged that utilize the knowledge contained in foundational models such as large-scale language models (hereinafter referred to as "LLMs"). This allows robots to generate trajectories of actions they should perform, taking instructions given by humans and surrounding conditions such as images as input, enabling them to perform a variety of tasks. For example, Non-Patent Document 1 discloses a technology that uses a visual language model (hereinafter referred to as "VLM") to learn robot work data and large-scale web data, thereby generating robot movements expressed in text in an end-to-end manner, using human instructions and the current image as input. Non-patent document 2 discloses a technique for generating robot trajectories using LLM (Limited Literacy Modeling) in a zero-shot (no additional learning) manner. Non-patent document 3 discloses a technique for generating a trajectory that adheres to the constraints of the arm's range of motion by using diffusion models that have different roles in estimating the tip position of a robot arm and generating joint angles, and then integrating these two models. Figure 1 illustrates the first requirement for generating optimal behavior that satisfies multiple constraints.Figure 2 is a block diagram showing an example of the functional configuration of a robot control device to which a robot control method according to an embodiment of the present invention is applied.Figure 3 is a conceptual diagram showing the processing flow by a robot control device according to an embodiment of the present invention.Figure 4 is a simplified diagram showing the hardware configuration of a typical computer. Embodiments of the present invention will be described below with reference to the drawings. The drawings are schematic or conceptual. In this specification and in each drawing, elements similar to those described in previously shown drawings are denoted by the same reference numerals, and detailed or redundant explanations are omitted as appropriate. As mentioned above, the objective of the robot control device, robot control method, and program of the present invention is to generate optimal movements that adhere to multiple constraints according to the work context. Here, constraints refer to the things that must be observed when the robot performs a task. Furthermore, optimal movements refer to movements that efficiently perform the task. For example, even if a task can be reliably performed, if the robot moves very slowly and cannot complete the task within the expected time, it cannot be considered an optimal movement. To achieve this objective, the following three requirements are necessary. The first requirement is the ability to estimate complex constraints from the current work situation and convert them into a format that can be handled by the model. Figure 1 illustrates the first requirement for generating optimal behavior that satisfies multiple constraints. Examples of complex constraints include, for instance, as illustrated in Figure 1, when instructing a robot 20 to move a glass 30 filled with water, there are speed constraints (Constraint 1) requiring the glass 30's speed to be kept as constant as possible, and state constraints (Constraint 2) requiring the movement trajectory to be generated within the robot 20's range of motion. The second requirement is the ability to handle unfamiliar combinations of constraints not included in the training data. The third requirement is the ability to flexibly respond to constraints of varying importance. For example, using the velocity and state constraints explained in the first requirement, the state constraint—generating a trajectory within the robot's range of motion—is an absolute requirement, while the velocity constraint—keeping the cup's speed as constant as possible—is a constraint that can be compromised. To meet these requirements, a robot control device to which the robot control method according to the embodiment of the present invention is applied interprets the work situation, for example, using LLM/VLM, estimates multiple constraints to be observed and their importance, and then generates a trajectory under multiple constraints based on a diffusion model (expressed as Box constraints and importance). This expresses general constraints as "multiple combinations of basic constraints" and their "importance," and ensures their adherence. Figure 2 is a block diagram showing an example of the functional configuration of a robot control device to which the robot control method according to an embodiment of the present invention is applied. The robot control device 10 includes a destination estimator 12, a multiple constraint representat