US-12617081-B2 - Mobile manipulator robot and method for using the same

Abstract

Disclosed are mobile manipulator and methods of using the mobile manipulator, there the mobile manipulator includes an autonomous mobile robot (AMR) comprising a LiDAR sensor, a camera sensor, and a moving member, a manipulator robot comprising a torque sensor, a current sensor, and an image sensor, and a processor is configured to determine, via the AMR, whether a worker is approaching, determine, via the manipulator robot, whether the worker interferes with an operation of the mobile manipulator robot, activate a touching mode, in response to the determining of the interference by the worker, and analyze a force for a robot operation based on touching to operate at least one of the AMR or the manipulator robot.

Inventors

• Il Yong EOM
• Young Gi JUNG
• Hyeon Guk Kim
• Ju Mong LEE

Assignees

• HYUNDAI MOBIS CO., LTD.

Dates

Publication Date

20260505

Application Date

20230705

Priority Date

20220707

Claims (18)

1. 1 . A mobile manipulator robot system comprising: an autonomous mobile robot (AMR) comprising a LiDAR sensor, a camera sensor, and a moving member; a manipulator robot comprising a torque sensor, a current sensor, and an image sensor; and a processor is configured to: determine, via the AMR, whether a worker is approaching; determine, via the manipulator robot, whether the worker interferes with an operation of the mobile manipulator robot; activate a touching mode, in response to the determining of the interference by the worker; analyze a force for a robot operation based on touching to operate at least one of the AMR or the manipulator robot; recognize the worker via the AMR to determine whether the worker is approaching; determine, via the manipulator robot, whether the worker collides with the mobile manipulator robot; activate a safe mode in response to the collision with the worker; analyze an avoidance direction based on the collision to control at least one of the AMR of the manipulator robot; calculate the avoidance direction of the manipulator robot, in response to determining that the worker collides with the manipulator robot; control the manipulator robot to perform an avoidance operation, in response to the avoidance direction being calculated; determine whether the mobile manipulator robot will collide with the worker, based on the avoidance operation of the manipulator robot; and control the AMR and the manipulator robot to simultaneously perform the avoidance operation, in response to the mobile manipulator robot colliding with the worker.
2. 2 . The mobile manipulator robot system of claim 1 , wherein the processor is further configured to: recognize a contact of the worker via sensor information received from the manipulator robot, in response to the worker gripping the manipulator robot; and determine that the interference with the worker has occurred, in response to the sensor information received from the manipulator robot exceeding a threshold.
3. 3 . The mobile manipulator robot system of claim 1 , wherein the processor is further configured to: determine a location of contact between the manipulator robot and the worker; determine a strength of the force needed for the robot operation; determine a direction of the force needed for the robot operation; and determine an applied time period of the force needed for the robot operation.
4. 4 . The mobile manipulator robot system of claim 3 , wherein the processor is further configured to: determine whether to control all axes or to control some axes of the manipulator robot based on the location of the contact; and control the AMR and the manipulator robot to operate simultaneously, in response to determining to control all of the axes.
5. 5 . The mobile manipulator robot of claim 4 , wherein the processor is further configured to control the AMR or the manipulator robot to operate, in response to determining to control some of the axes.
6. 6 . The mobile manipulator robot system of claim 3 , wherein the processor is further configured to control only the manipulator robot to operate, in response to determining the strength of the force for the robot operation is smaller than a first range.
7. 7 . The mobile manipulator robot system of claim 3 , wherein the processor is further configured to control the manipulator robot and the AMR to operate simultaneously, in response to determining the strength of the force for the robot operation is equal to or greater than a first range and is equal to or lesser than a second range.
8. 8 . The mobile manipulator robot system of claim 3 , wherein the processor is further configured to: control the AMR to operate by projecting the direction of the force for the robot operation to a XZ plane; and control the manipulator robot to operate by projecting the direction of the force for the robot operation to a XY plane.
9. 9 . The mobile manipulator robot system of claim 3 , wherein the processor is further configured to control the mobile manipulator robot to operate by the touching, in response to the force for the robot operation being generated and maintained for a time period or more.
10. 10 . The mobile manipulator robot system of claim 1 , wherein the processor is further configured to: determine whether a distance sensed by the LiDAR sensor is within a threshold area; and distinguish the worker using the image sensor when the distance is within the threshold area.
11. 11 . The mobile manipulator robot system of claim 1 , wherein the processor is further configured to generate vector coordinates in an opposite direction of an impact caused by the collision between the mobile manipulator robot and the worker.
12. 12 . The mobile manipulator robot system of claim 11 , wherein the processor is further configured to determine whether the vector coordinates are beyond a movement limit of the manipulator robot.
13. 13 . The mobile manipulator robot system of claim 1 , wherein the processor is further configured to control the mobile manipulator robot to return to a state before the collision after the avoidance operation.
14. 14 . The mobile manipulator robot system of claim 13 , wherein the processor is further configured to: store location information of each of the manipulator robot and the AMR at a moment when the collision occurs; determine whether the collision situation is released by the avoidance operation; control at least one of the manipulator robot or the AMR to move to the stored location, in response to the collision situation is released by the avoidance operation.
15. 15 . The mobile manipulator robot system of claim 14 , wherein the processor is further configured to: determine whether a magnitude of an impact caused by the collision is equal to or smaller than a threshold; determine whether the impact is applied within a predetermined time period, in response to the magnitude of the impact being equal to or smaller than the threshold; and determine that the collision situation is released, in response to the impact being applied within the predetermined time period.
16. 16 . The mobile manipulator robot system of claim 13 , wherein the processor is further configured to: check a location before when the AMR moves before the collision; check the location via the manipulator robot corresponding to a feature point of a worktable; and correct the location of the mobile manipulator robot via calibration.
17. 17 . A processor-implemented method for controlling a mobile manipulator robot including an autonomous mobile robot (AMR) and a manipulator robot, the method comprising: determining, via the AMR, whether a worker is approaching; determining, via the manipulator robot, whether the worker interferes with an operation of the mobile manipulator robot; activating a touching mode, in response to the determining of the interference by the worker; and analyzing a force for a robot operation based on touching to operate at least one of the AMR or the manipulator robot; recognizing the worker via the AMR to determine whether the worker is approaching; determining, via the manipulator robot, whether the worker collides with the mobile manipulator robot; activating a safe mode in response to the collision with the worker; analyzing an avoidance direction based on the collision to control at least one of the AMR or the manipulator robot; calculating the avoidance direction of the manipulator robot, in response to determining that the worker collides with the manipulator robot; controlling the manipulator robot to perform an avoidance operation, in response to the avoidance direction being calculated; determining whether the mobile manipulator robot will collide with the worker, based on the avoidance operation of the manipulator robot; controlling the AMA and the manipulator robot to simultaneously perform the avoidance operation, in response to the mobile manipulator robot colliding with the worker.
18. 18 . A processor-implemented method for safely controlling a mobile manipulator robot including an autonomous mobile robot (AMR) and a manipulator robot, the method comprising: determining, via the AMR, whether a worker is approaching; determining, via the manipulator robot, whether the worker collides with the mobile manipulator robot; activating a safe mode in response to the collision with the worker; and analyzing an avoidance direction based on the collision to control at least one of the AMR or the manipulator robot; recognizing the worker via the AMR to determine whether the worker is approaching; determining, via the manipulator robot, whether the worker collides with the mobile manipulator robot; activating a safe mode in response to the collision with the worker; analyzing an avoidance direction based on the collision to control at least one of the AMR or the manipulator robot; calculating the avoidance direction of the manipulator robot, in response to determining that the worker collides the manipulator robot; controlling the manipulator robot to perform an avoidance operation, in response to the avoidance direction being calculated; determining whether the mobile manipulator robot will collide with the worker, based on the avoidance operation of the manipulator robot; and controlling the AMA and the manipulator robot to simultaneously perform the avoidance operation, in response to the mobile manipulator robot colliding with the worker.

Description

CROSS REFERENCE TO RELATED APPLICATION This application claims the benefit under 35 U.S.C. § 119(a) of Korean Patent Application Nos. 10-2022-0083869, filed on Jul. 7, 2022 and 10-2022-0083870, filed on Jul. 7, 2022, in the Korean Intellectual Property Office, the entire disclosure of which are incorporated herein by reference for all purposes. BACKGROUND 1. Field The present disclosure relates to a mobile manipulator robot, and specifically, to a teaching method using an autonomous mobile robot (AMR) and a manipulator robot. 2. Discussion of Related Art Recently, as the number of factories in a form of smart factories increases, a mobile robot in a form of an automatic guided vehicle (AGV) that moves along a guide used in an existing factory or a warehouse has been proposed and used. Because the AGV is limited to moving along the guide on a predetermined path within the smart factory, research on an autonomous mobile robot (AMR)-type mobile robot that does not require the guide and is flexibly applicable to a field situation is increasing recently. In addition, as a fixed-type manipulator robot that has been used for assembly, palletizing, and the like is mounted on the autonomous mobile robot (AMR) and moves, various types of work are being attempted. Therefore, in operating a mobile manipulator, which is a combination of the AMR and the manipulator robot, a method for easily operating a robot that may be touched by a worker is needed. SUMMARY This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. In one general aspect, there is provided a mobile manipulator robot including an autonomous mobile robot (AMR) comprising a LiDAR sensor, a camera sensor, and a moving member, a manipulator robot comprising a torque sensor, a current sensor, and an image sensor, and a processor is configured to determine, via the AMR, whether a worker is approaching, determine, via the manipulator robot, whether the worker interferes with an operation of the mobile manipulator robot, activate a touching mode, in response to the determining of the interference by the worker, and analyze a force for a robot operation based on touching to operate at least one of the AMR or the manipulator robot. The processor may be configured to recognize a contact of the worker via sensor information received from the manipulator robot, in response to the worker griping the manipulator robot, and determine that the interference with the worker has occurred, in response to the sensor information received from the manipulator robot exceeding a threshold. The processor may be configured to determine a location of contact between the manipulator robot and the worker, determine a strength of the force needed for the robot operation, determine a direction of the force needed for the robot operation, and determine an applied time period of the force needed for the robot operation. The processor may be configured to determine whether to control all axes or to control some axes of the manipulator robot based on the location of the contact, and control the AMR and the manipulator robot to operate simultaneously, in response to determining to control all of the axes. The processor may be configured to control the AMR or the manipulator robot to operate, in response to determining to control some of the axes. The processor may be configured to control only the manipulator robot to operate, in response to determining the strength of the force for the robot operation is smaller than a first range. The processor may be configured to control the manipulator robot and the AMR to operate simultaneously, in response to determining the strength of the force for the robot operation is equal to or greater than a first range and is equal to or lesser than a second range. The processor may be configured to control the AMR to operate by projecting the direction of the force for the robot operation to a XZ plane, and control the manipulator robot to operate by projecting the direction of the force for the robot operation to a XY plane. The processor may be configured to control the mobile manipulator robot to operate by the touching, in response to the force for the robot operation being generated and maintained for a time period or more. The processor may be configured to recognize the worker via the AMR to determine whether the worker is approaching, determine, via the manipulator robot, whether the worker collides with the mobile manipulator robot, activate a safe mode in response to the collision with the worker, and analyze an avoidance direction based on the collision to control at least one of the AMR or the manipulator robot. The processor may be configured to determine whethe