US-12625496-B2 - Mobile robot for transporting article in manufacturing facility and article transport system including same

US12625496B2US 12625496 B2US12625496 B2US 12625496B2US-12625496-B2

Abstract

A method of operating a mobile robot for transporting an article in a manufacturing facility, the mobile robot, and an article transport system including the same are proposed. The method of operating the mobile robot includes obtaining a 3-D depth image, extracting, from the 3-D depth image, a region of interest corresponding to a traveling path of the mobile robot on a bottom surface of the manufacturing facility, generating a projected point cloud by projecting an object detected from the region of interest on a reference plane corresponding to the bottom surface, generating an imaginary point cloud filled with voxels in the reference plane, detecting a hole existing in the bottom surface by comparing the imaginary point cloud to the projected point cloud, and travelling while avoiding the hole.

Inventors

Seung Seok HA
In Sung Choi
Seung Jun Lee

Assignees

SEMES CO., LTD.

Dates

Publication Date: 20260512
Application Date: 20221220
Priority Date: 20211221

Claims (6)

1 . A method of operating a mobile robot for transporting an article in a manufacturing facility, the method comprising: obtaining a three dimensional (3-D) depth image from a depth camera; extracting, from the 3-D depth image, a region of interest corresponding to a traveling path of the mobile robot on a bottom surface of the manufacturing facility; generating a projected point cloud by projecting an object detected from the region of interest on a reference plane corresponding to the bottom surface; generating an imaginary point cloud representing an artificial uninterrupted bottom surface filled with voxels in the reference plane, wherein the imaginary point cloud is independent from the 3-D depth image obtained from the depth camera; detecting a hole existing in the bottom surface by comparing the imaginary point cloud to the projected point cloud; and travelling while avoiding the hole, wherein the generating of the projected point cloud comprises: projecting voxels in the region of interest of the 3-D depth image on the reference plane; and applying a voxel filter with respect to the projected voxels to generate the projected point cloud, wherein the applying of the voxel filter includes reducing a number of the voxels located in the reference plane, wherein when a number of voxels located in a unit area in the reference plane is equal to or greater than a threshold value, the voxel filter changes the voxels located in the unit area into one voxel, wherein when the number of the voxels located in the unit area in the reference plane is less than the threshold value, the voxel filter removes the voxels in the unit area, wherein the generating of the imaginary point cloud comprises generating the imaginary point cloud filled with first imaginary voxels in the unit area of the voxel filter in the reference plane, wherein the detecting of the hole existing in the bottom surface comprises: generating a remaining point cloud including second imaginary voxels by removing overlapping voxels of the projected point cloud from the first imaginary voxels of the imaginary point cloud; and detecting, from the reference plane, a region with the second imaginary voxels of the remaining point cloud as a region of the hole, and wherein the traveling while avoiding the hole comprises: changing the second imaginary voxels in the region of the hole into an obstacle object in the 3-D depth image by applying reversal with respect to the region with the second imaginary voxels of the remaining point cloud in the reference plane; and presetting a moving path so that the mobile robot travels to a target location while avoiding the obstacle object.
2 . The method of claim 1 , wherein the extracting of the region of interest comprises presetting a region corresponding to the traveling path of the mobile robot within a lower bottom detection range based on a reference height in the 3-D depth image, into the region of interest.
3 . A mobile robot for transporting an article in a manufacturing facility, the mobile robot comprises: a robot main body travelling in an internal space of the manufacturing facility; a depth camera provided in the robot main body; and a processor configured to control travelling of the robot main body, wherein the processor is configured to: obtain a 3-D depth image from the depth camera, extract a region of interest, which corresponds to a travelling path of the mobile robot on a bottom surface of the manufacturing facility, from the 3-D depth image, generate a projected point cloud by projecting an object detected from the region of interest on a reference plane corresponding to the bottom surface, generate an imaginary point cloud representing an artificial uninterrupted bottom surface filled with voxels in the reference plane, wherein the imaginary point cloud is independent from the 3-D depth image obtained from the depth camera, detect a hole existing on the bottom surface by comparing the imaginary point cloud to the projected point cloud, and control the robot main body so that the robot main body travels while avoiding the hole, and wherein the processor is further configured to: project voxels in the region of interest of the 3-D depth image on the reference plane; generate the projected point cloud by applying a voxel filter with respect to the projected voxels, wherein the voxel filter is a filter configured to reduce a number of the voxels located in the reference plane, wherein when the number of voxels located in a unit area in the reference plane is equal to or greater than a threshold value, the voxel filter changes the voxels located in the unit area into one voxel, and when the number of the voxels located in the unit area in the reference plane is less than the threshold value, the voxel filter removes the voxels in the unit area; generate the imaginary point cloud filled with a first imaginary voxels in the unit area of the voxel filter of the reference plane; generate a remaining point cloud including second imaginary voxels by removing overlapping voxels of the projected point cloud from the first imaginary voxels the imaginary point cloud; detect, from the reference plane, a region with the second imaginary voxels of the remaining point cloud as a region of the hole; change the second imaginary voxels of the region of the hole into an obstacle object in the 3-D depth image by applying reversal with respect to the region with the second imaginary voxels of the remaining point cloud in the reference plane; and preset a moving path of the robot main body so that the robot main body travels to a target location while avoiding the obstacle object.
4 . The mobile robot for claim 3 , wherein the processor is further configured to preset a region corresponding to the travelling path of the mobile robot in a lower bottom detection range of a reference height in the 3-D depth image, into the region of interest.
5 . An article transport system in a manufacturing facility, the article transport system comprising: a system controller configured to transmit an instruction for transporting an article in the manufacturing facility; and a mobile robot configured to transport the article according to the instruction, wherein the mobile robot comprises: a robot main body travelling in an internal space of the manufacturing facility; a depth camera provided in the robot main body and configured to generate a 3-D depth image; and a processor configured to control travelling of the robot main body, to detect an obstacle located on a travelling path of the mobile robot from an upper region of interest of the 3-D depth image, and to detect a hole located on a bottom surface of the manufacturing facility from a lower region of interest of the 3-D depth image, wherein the processor is configured to: generate a projected point cloud by projecting an object detected from the lower region of interest to a reference plane corresponding to the bottom surface, generate an imaginary point cloud representing an artificial uninterrupted bottom surface filled with voxels in the reference plane, wherein the imaginary point cloud is independent from the 3-D depth image obtained from the depth camera, detect the hole existing on the bottom surface by comparing the imaginary point cloud and the projected point cloud, and control the robot main body so that the robot main body travels while avoiding the hole, and wherein the processor is further configured to: project voxels of the lower region of interest of the 3-D depth image, on the reference plane; generate the projected point cloud by applying a voxel filter with respect to the projected voxels, wherein the voxel filter is a filter configured to reduce the number of voxels located in the reference plane, wherein when a number of the voxels located in a unit area in the reference plane is equal to or greater than a threshold value, the voxel filter changes the voxels located in the unit area into one voxel, and when the number of the voxels located in the unit area in the reference plane is less than the threshold value, the voxel filter removes the voxels in the unit area; generate the imaginary point cloud filled with a first imaginary voxels in the unit area of the voxel filter of the reference plane; generate a remaining point cloud including second imaginary voxels by removing overlapping voxels of the projected point cloud, from the first imaginary voxels of the imaginary point cloud; detect, from the reference plane, a region with the second imaginary voxels of the remaining point cloud as a region of the hole; change the second imaginary voxels of the region of the hole into an obstacle object in the 3-D depth image by applying reversal with respect to the region with the voxels of the remaining point cloud in the reference plane; and preset a moving path of the robot main body so that the robot main body travels to a target location while avoiding the obstacle object.
6 . The article transport system of claim 5 , wherein the mobile robot is configured to transmit location information of the hole to the system controller, and wherein the system controller is configured to store the location information of the hole and transmit the location information of the hole to another mobile robot.

Description

CROSS REFERENCE TO RELATED APPLICATION The present application claims priority to Korean Patent Application No. 10-2021-0183510, filed Dec. 21, 2021, the entire contents of which is incorporated by reference herein for all purposes. BACKGROUND OF THE INVENTION Field of the Invention The present disclosure relates to a method of operating a mobile robot for transporting an article in a manufacturing facility, the mobile robot, and an article transport system including the mobile robot. Description of the Related Art A semiconductor or display manufacturing process is a process of manufacturing a final product by performing several tens to hundreds of processes on a substrate (wafer or glass), and each process may be performed by a manufacturing facility that performs the corresponding process. When a process in a specific manufacturing facility is completed, in order to proceed with a next process, an article (substrate) may be transported to a next manufacturing facility and may be stored in a storage facility for a predetermined period. An article transport system refers to a system allowing the article to be transported or stored for the manufacturing process as described above, and mainly, may be divided into a transport system for transporting the article and a storage system for storing the article. In the article transport system, not only an overhead hoist transport (OHT) travelling along a rail installed on a ceiling but also a mobile robot transporting the article while travelling along a bottom surface, such as an anonymous driving vehicle, has been introduced. Since there is a risk of a crash to various obstacles when the mobile robot travels in the internal space of a manufacturing facility, various techniques for avoiding the obstacles by using cameras, radar, or lidar have been introduced. However, a general obstacle detection technology is focused on a method of detecting an obstacle object located in the surrounding, but in the case of semiconductor manufacturing facility, since there are cases where holes are formed in the bottom surface, a method of detecting and avoiding the hole formed on the bottom surface as well as the obstacle object is required. SUMMARY OF THE INVENTION Accordingly, an embodiment of the present disclosure is intended to provide a method of operating a mobile robot capable of detecting a hole formed in a bottom surface of a manufacturing facility, the mobile robot, and an article transport system including the same. The technical problem of the present disclosure is not limited to the above-mention, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. According to an embodiment of the present disclosure, a method of operating a mobile robot for transporting an article in a manufacturing facility includes: obtaining a 3-D depth image; extracting, from the 3-D depth image, a region of interest corresponding to a traveling path of the mobile robot on a bottom surface of the manufacturing facility; generating a projected point cloud by projecting an object detected from the region of interest on a reference plane corresponding to the bottom surface; generating an imaginary point cloud filled with voxels in the reference plane; detecting a hole existing in the bottom surface by comparing the imaginary point cloud to the projected point cloud; and travelling while avoiding the hole. The extracting of the region of interest may include presetting a region corresponding to the traveling path of the mobile robot within a lower bottom detection range based on a reference height in the 3-D depth image, into the region of interest. The generating of the projected point cloud may include: projecting voxels in the region of interest of the 3-D depth image on the reference plane; and applying a voxel filter with respect to the projected voxels to generate the projected point cloud. The voxel filter may be a filter configured to reduce the number of voxels located in the reference plane. When the number of voxels located in a unit area in the reference plane is equal to or greater than a threshold value, the voxel filter may change the voxels located in the unit area into one voxel, and when the number of the voxels located in the unit area in the reference plane is less than the threshold value, the voxel filter may remove the voxels in the unit area. The generating of the imaginary point cloud may include generating the imaginary point cloud filled with voxels in the unit area of the voxel filter in the reference plane. The detecting of the hole existing in the bottom surface may include: generating a remaining point cloud by removing voxels overlapping with the projected point cloud from the imaginary point cloud; and detecting, from the reference plane, a region with voxels of the remaining point cloud as a region of the hole. The traveling while avoiding the hole may include: changing voxels in the region of the hol