EP-4741114-A1 - CONTROL DEVICE, INFORMATION PROCESSING DEVICE, AND METHOD

EP4741114A1EP 4741114 A1EP4741114 A1EP 4741114A1EP-4741114-A1

Abstract

A control device for causing a robot to perform a predetermined operation, includes: a communication interface configured to perform data communication with an external device; and a controller configured to control the robot by data communication via the communication interface. A plurality of partial trajectories segmenting a trajectory are set, the trajectory representing motion of the robot during the predetermined operation. The plurality of partial trajectories include a first partial trajectory and a second partial trajectory adjacent to each other in the trajectory. The first partial trajectory and the second partial trajectory include an overlapping portion where the first partial trajectory and the second partial trajectory overlap with each other, a first individual portion only belonging to the first partial trajectory, and a second individual portion only belonging to the second partial trajectory. The controller is configured to cause the robot to perform the predetermined operation in the overlapping portion in accordance with both of the first partial trajectory and the second partial trajectory.

Inventors

OKUMURA, RYO

Assignees

Panasonic Intellectual Property Management Co., Ltd.

Dates

Publication Date: 20260513
Application Date: 20240621

Claims (20)

A control device for causing a robot to perform a predetermined operation, the control device comprising: a communication interface configured to perform data communication with an external device; and a controller configured to control the robot by data communication via the communication interface, wherein a plurality of partial trajectories segmenting a trajectory are set, the trajectory representing motion of the robot during the predetermined operation, the plurality of partial trajectories include a first partial trajectory and a second partial trajectory adjacent to each other in the trajectory, the first partial trajectory and the second partial trajectory include an overlapping portion where the first partial trajectory and the second partial trajectory overlap with each other, a first individual portion only belonging to the first partial trajectory, and a second individual portion only belonging to the second partial trajectory, and the controller is configured to cause the robot to perform the predetermined operation in the overlapping portion in accordance with both of the first partial trajectory and the second partial trajectory.
The control device according to claim 1, wherein the controller is configured to cause the robot to perform the predetermined operation in the first individual portion in accordance with the first partial trajectory, and the robot to perform the predetermined operation in the second individual portion in accordance with the second partial trajectory.
The control device according to claim 1, further comprising: an information input interface via which information on the plurality of partial trajectories is input; and a trainer configured to perform training in machine learning for causing the robot to perform the predetermined operation, in accordance with the plurality of partial trajectories set based on the information input via the information input interface, wherein the trainer is configured to perform the training in the machine learning in accordance with the first individual portion, the overlapping portion, and the second individual portion, in the first partial trajectory and the second partial trajectory.
The control device according to any one of claims 1 to 3, further comprising an information processor configured to control machine learning for causing the robot to perform the predetermined operation, wherein the information processor is configured to: cause a display that displays information, to display the trajectory representing the motion of the robot during the predetermined operation; cause an input interface that receives a user operation, to receive the user operation on the trajectory being displayed on the display to set the plurality of partial trajectories that segment the trajectory; and control the machine learning to cause the robot to perform the predetermined operation for each of the plurality of partial trajectories, in accordance with set partial trajectories based on the user operation, the set partial trajectories, as the plurality of partial trajectories that segment the trajectory, have an overlapping portion between partial trajectories that are adjacent to each other in the trajectory, and the controller is configured to control the robot in accordance with a result of the machine learning controlled by the information processor.
An information processing device for controlling machine learning to cause a robot to perform a predetermined operation, the information processing device comprising: a display configured to display information; an input interface configured to receive a user operation; and a controller configured to control the display and the input interface, wherein the controller is configured to: cause the display to display a trajectory representing motion of the robot during the predetermined operation; cause the input interface to receive the user operation on the trajectory being displayed on the display to set a plurality of partial trajectories that segment the trajectory; and control the machine learning to cause the robot to perform the predetermined operation for each of the plurality of partial trajectories, in accordance with set partial trajectories based on the user operation, and wherein the set partial trajectories, as the plurality of partial trajectories that segment the trajectory, have an overlapping portion between partial trajectories that are adjacent to each other in the trajectory.
The information processing device according to claim 5, wherein the controller is configured to control the machine learning in accordance with the set partial trajectories, to cause the robot to perform a plurality of partial operations respectively corresponding to the set partial trajectories in the predetermined operation.
The information processing device according to claim 5, wherein the controller is configured to detect whether the plurality of partial trajectories partially overlap with each other, based on input partial trajectories by the user operation.
The information processing device according to claim 7, wherein the controller is configured to sets the plurality of partial trajectories such that each partial trajectory partially overlaps with an adjacent partial trajectory, when the controller detects that the plurality of partial trajectories do not partially overlap with each other.
The information processing device according to claim 5, wherein the controller is configured to: cause the input interface to receive an additional user operation to set a lower limit of an overlap amount indicating a degree by which the plurality of partial trajectories overlap with each other, and when the overlap amount in the plurality of partial trajectories is less than the lower limit, set the plurality of partial trajectories to increase the overlap amount to the lower limit or more.
The information processing device according to claim 5, wherein the controller is configured to: assign attribute information identifying the plurality of partial trajectories from one another, to each of the set partial trajectories based on the user operation; and cause the display to display the attribute information assigned to each of the partial trajectories to indicate that a plurality of pieces of the attribute information are assigned to the overlapping portion where the plurality of partial trajectories overlap with each other in the trajectory.
The information processing device according to claim 10, wherein the controller is configured to control the machine learning to cause the robot to perform the predetermined operation, in accordance with the attribute information assigned to the plurality of partial trajectories.
The information processing device according to claim 10, wherein the controller is configured to: cause the display to display each of the plurality of partial trajectories in a different manner according to the attribute information assigned to the partial trajectory; and display, in the overlapping portion where the plurality of partial trajectories overlap with each other, overlapping partial trajectories to be superimposed on one another in a manner according to the attribute information of the respective overlapping partial trajectories.
The information processing device according to claim 5, wherein the display is configured to display a direction of the motion of the robot in the predetermined operation after the machine learning, and the controller is configured to update display of the direction of the motion on the display as a result of controlling the machine learning in accordance with the set partial trajectories based on the user operation.
The information processing device according to claim 5, wherein the controller is configured to acquire, as the trajectory, coordinate data representing the motion of the robot in increments of time, the motion being resultant of direct teaching.
The information processing device according to claim 5, wherein the controller is configured to acquire, as the trajectory, coordinate data representing the motion of the robot in increments of time, the motion being resultant of remote control.
The information processing device according to claim 5, wherein the controller is configured to display a marker indicating a position where the trajectory satisfies a predetermined condition, on the trajectory displayed on the display, the predetermined condition being associated with a change in an operation of the robot.
The information processing device according to claim 5, wherein the controller is configured to: train, in the machine learning, a control model that controls an operation of the robot, in accordance with the plurality of partial trajectories set based on the user operation; and cause the control model to output a control command to operate the robot, by sequentially inputting positions or postures at which the robot operates, to cause the robot to perform the predetermined operation.
The information processing device according to claim 17, wherein the control command output from the control model includes an average value and a variance value that define a probability distribution of a relative displacement to a target coordinate where the robot is caused to operate at an input position or posture, the control command being output in response to the input position or posture.
An information processing method for controlling machine learning to cause a robot to perform a predetermined operation, the information processing method comprising, by a controller of a computer controlling a display that displays information and an input interface that receives a user operation: causing the display to display a trajectory representing motion of the robot during the predetermined operation; causing the input interface to receive the user operation on the trajectory being displayed on the display to set a plurality of partial trajectories that segment the trajectory; and controlling the machine learning to cause the robot to perform the predetermined operation for each of the plurality of partial trajectories, in accordance with set partial trajectories based on the user operation, wherein the set partial trajectories, as the plurality of partial trajectories that segment the trajectory, have an overlapping portion between partial trajectories that are adjacent to each other in the trajectory.
A program for causing the controller of the computer to execute the information processing method according to claim 19.

Description

TECHNICAL FIELD The present disclosure relates to a control device and a method for causing a robot to perform a predetermined operation, and an information processing device and a method for controlling machine learning causing a robot to perform a predetermined operation. BACKGROUND ART WO 2016/103307 A discloses a method for generating an operation program for a dual-arm robot. This method specifies, with a user input using a GUI, a variable for a template element operation program for causing a robot to perform each element work forming the whole work of the robot. The template element operation program is configured to include, as variables, one or more finger position coordinates (teaching points) specifying a robot motion (position changing and orientation changing) required for the element operation, and specify the robot motion by specifying the all finger position coordinates. Thus, by merely specifying the parameters of the template element operation program by using the GUI, a teacher who teaches a motion to the robot can generate a motion program that relates to movements/motions and that contains hand end position coordinates of the robot. Patent Document Patent Document 1: WO 2016/103307 A Non-Patent Document Non-Patent Document 1: G. Franzese, A. Meszaros, L. Peternel and J. Kober, "ILoSA: Interactive learning of stiffness and attractors," in IROS, 2021. SUMMARY The present disclosure provides a control device, an information processing device, and a method that can facilitate operating a robot accurately. A control device according to one aspect of the present disclosure causes a robot to perform a predetermined operation. The control device includes a communication interface that performs data communication with an external device, and a controller that controls the robot by data communication via the communication interface. A plurality of partial trajectories segmenting a trajectory are set, the trajectory representing motion of the robot during the predetermined operation. The plurality of partial trajectories include a first partial trajectory and a second partial trajectory adjacent to each other in the trajectory. The first partial trajectory and the second partial trajectory include: an overlapping portion where the first partial trajectory and the second partial trajectory overlap with each other; a first individual portion only belonging to the first partial trajectory; and a second individual portion only belonging to the second partial trajectory. The controller causes the robot to perform the predetermined operation in the overlapping portion in accordance with both of the first partial trajectory and the second partial trajectory. An information processing device according to an aspect of the present disclosure controls machine learning for causing a robot to perform a predetermined operation. The information processing device includes a display that displays information, an input interface that receives a user operation, a controller that controls the display and the input interface. The controller causes the display to display a trajectory representing motion of the robot during the predetermined operation. The controller causes the input interface to receive the user operation on the trajectory being displayed on the display to set a plurality of partial trajectories that segment the trajectory. The controller controls the machine learning to cause the robot to perform the predetermined operation for each of the plurality of partial trajectories, in accordance with set partial trajectories based on the user operation. The set partial trajectories, as the plurality of partial trajectories that segment the trajectory, have an overlapping portion between partial trajectories that are adjacent to each other in the trajectory. These general and specific aspects may be implemented as a system, a method, and a computer program, and a combination thereof. With the control device, the information processing device, and the method according to the present disclosure, it is possible to facilitate operating the robot accurately. BRIEF DESCRIPTION OF DRAWINGS Fig. 1 is a diagram for explaining an outline of a control system according to a first embodiment;Fig. 2 is a block diagram illustrating a configuration of a robot, a robot control device, and a terminal device in the control system;Figs. 3A and 3B are diagrams for explaining a problem in training of a control model using a trajectory of the robot;Fig. 4 is a diagram for explaining an operation of the control system;Fig. 5 is a sequence diagram illustrating an entire operation of the control system;Fig. 6 is a flowchart illustrating trajectory segmentation processing in the terminal device according to the first embodiment;Figs. 7A and 7B are diagrams illustrating an example of display for the trajectory segmentation processing in the terminal device according to the first embodiment;Figs. 8A and 8B are diagrams illustrating an exa