US-12623856-B2 - Method for determining material-cage stacking, computer device, and storage medium

Abstract

A method for determining material-cage stacking, a computer device, and a storage medium are provided. The method includes the following. A material-cage image is obtained by photographing a first stacking apparatus of a first material cage and a second stacking apparatus of a second material cage. The stacking apparatuses of the two material cages in the material-cage image can be recognized respectively with two detection models. The first stacking result is obtained by obtaining location information of the stacking apparatuses of the two material cages with the first detection model, and the second stacking result is obtained with the second detection model.

Inventors

• Peng Chen
• Mu FANG
• Yujie LU
• Bingchuan YANG

Assignees

• VISIONNAV ROBOTICS USA INC.

Dates

Publication Date

20260512

Application Date

20240607

Priority Date

20210610

Claims (20)

1. 1 . A method for determining material-cage stacking, comprising: obtaining a material-cage image by photographing a first stacking apparatus of a first material cage and a second stacking apparatus of a second material cage; performing first target detection on the material-cage image with a first detection model to recognize the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage; determining first location information of the first stacking apparatus and second location information of the second stacking apparatus, and determining a first stacking result based on the first location information and the second location information; performing second target detection on the material-cage image with a second detection model to extract feature information of the material-cage image, and obtaining a second stacking result based on the feature information; and determining whether the first material cage is able to be stacked on the second material cage, based on the first stacking result and the second stacking result; wherein the first material cage has a first foot cup on a first surface of the first material cage and a first pier on a second surface of the first material cage, the second material cage has a second foot cup on a first surface of the second material cage and a second pier on a second surface of the second material cage, and wherein determining the first location information of the first stacking apparatus and the second location information of the second stacking apparatus, and determining the first stacking result based on the first location information and the second location information comprises: determining the first location information of the first foot cup in the first stacking apparatus and the second location information of the second pier in the second stacking apparatus; obtaining a distance between the first foot cup and the second pier in each of stacking groups based on the first location information and the second location information, wherein the first foot cup and the second pier in a same stacking group are aligned with each other in a vertical direction when the first material cage is stacked on the second material cage; for each of the stacking groups, obtaining a comparing result of the stacking group by comparing the distance between the first foot cup and the second pier with a distance threshold; and determining the first stacking result based on the comparing results of the stacking groups.
2. 2 . The method of claim 1 , wherein the material-cage image comprises a first material-cage image and a second material-cage image; and obtaining the material-cage image by photographing the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage comprises: obtaining the first material-cage image by photographing the first stacking apparatus in a first direction of the first material cage and the second stacking apparatus in the first direction of the second material cage; and obtaining the second material-cage image by photographing the first stacking apparatus in a second direction of the first material cage and the second stacking apparatus in the second direction of the second material cage, wherein the first direction is different from the second direction.
3. 3 . The method of claim 2 , wherein the first stacking apparatus and the second stacking apparatus each comprise foot cups and piers matched with the foot cups; obtaining the first material-cage image by photographing the first stacking apparatus in the first direction of the first material cage and the second stacking apparatus in the first direction of the second material cage comprises: obtaining the first material-cage image by photographing a foot cup in the first direction of the first material cage and a pier in the first direction of the second material cage; and obtaining the second material-cage image by photographing the first stacking apparatus in the second direction of the first material cage and the second stacking apparatus in the second direction of the second material cage comprises: obtaining the second material-cage image by photographing a foot cup in the second direction of the first material cage and a pier in the second direction of the second material cage.
4. 4 . The method of claim 3 , wherein the first direction and the second direction are two sides of a warehousing unmanned forklift.
5. 5 . The method of claim 1 , wherein performing the first target detection on the material-cage image with the first detection model to recognize the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage comprises: obtaining the feature information of the material-cage image with the first detection model, and recognizing the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage according to the feature information.
6. 6 . The method of claim 1 , wherein the first surface of the first material cage is a bottom surface of the first material cage, and the second surface of the first material cage is a top surface of the first material cage; the first surface of the second material cage is a bottom surface of the second material cage, and the second surface of the second material cage is a top surface of the second material cage; and the bottom surface of the first material cage is close to or in contact with the top surface of the second material cage when the first material cage is stacked on the second material cage.
7. 7 . The method of claim 1 , wherein determining the first location information of the first foot cup in the first stacking apparatus and the second location information of the second pier in the second stacking apparatus comprises: assigning a midpoint of a rectangular frame defined by the first foot cup and the second pier as a positioning point of the first foot cup and the second pier; and obtaining the first location information and the second location information in real time according to internal and external parameters of an image obtaining device and an equation of ground in a coordinate system of the image obtaining device.
8. 8 . The method of claim 1 , wherein determining the first stacking result based on the comparing results of the stacking groups comprises: determining the first stacking result is that the first material cage is able to be stacked on the second material cage, on condition that the comparing results of the stacking groups each indicate that the first material cage is able to be stacked on the second material cage; or determining the first stacking result to be each of the comparing results of the stacking groups.
9. 9 . The method of claim 1 , wherein before performing the second target detection on the material-cage image with the second detection model to extract the feature information of the material-cage image and obtaining the second stacking result based on the feature information, the method further comprises: obtaining a material-cage image sample, wherein the material-cage image sample comprises a first material-cage sample image of two material cages being able to be stacked and a second material-cage sample image of two material cages being not able to be stacked; and training the second detection model with the material-cage image sample and a label corresponding to the material-cage image sample and indicating whether two material cages are able to be stacked, to obtain a trained second detection model.
10. 10 . The method of claim 1 , wherein determining whether the first material cage is able to be stacked on the second material cage, based on the first stacking result and the second stacking result comprises: obtaining stacking data by weighting and summing the first stacking result and the second stacking result with a stacking determining model; and determining whether the first material cage is able to be stacked on the second material cage by comparing the stacking data with a stacking threshold.
11. 11 . The method of claim 10 , wherein the stacking determining model is a trained classifier.
12. 12 . The method of claim 1 , further comprising: triggering operations of stacking the first material cage on the second material cage upon determining that the first material cage is able to be stacked on the second material cage; or triggering operations of preventing the first material cage from being stacked on the second material cage upon determining that the first material cage is not able to be stacked on the second material cage.
13. 13 . The method of claim 1 , wherein the first location information is coordinate information of the first stacking apparatus, and the second location information is coordinate information of the second stacking apparatus.
14. 14 . A computer device, comprising: a processor; and a memory configured to store computer instructions which, when executed by the processor, enable the processor to: obtain a material-cage image by photographing a first stacking apparatus of a first material cage and a second stacking apparatus of a second material cage with an image obtaining device; perform first target detection on the material-cage image with a first detection model to recognize the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage; determine first location information of the first stacking apparatus and second location information of the second stacking apparatus, and determine a first stacking result based on the first location information and the second location information; perform second target detection on the material-cage image with a second detection model to extract feature information of the material-cage image, and obtain a second stacking result based on the feature information; and determine whether the first material cage is able to be stacked on the second material cage, based on the first stacking result and the second stacking result; wherein the first material cage has a first foot cup on a first surface of the first material cage and a first pier on a second surface of the first material cage, the second material cage has a second foot cup on a first surface of the second material cage and a second pier on a second surface of the second material cage, and wherein the processor configured to determine the first location information of the first stacking apparatus and the second location information of the second stacking apparatus, and determine the first stacking result based on the first location information and the second location information is configured to: determine the first location information of the first foot cup in the first stacking apparatus and the second location information of the second pier in the second stacking apparatus; obtain a distance between the first foot cup and the second pier in each of stacking groups based on the first location information and the second location information, wherein the first foot cup and the second pier in a same stacking group are aligned with each other in a vertical direction when the first material cage is stacked on the second material cage; for each of the stacking groups, obtain a comparing result of the stacking group by comparing the distance between the first foot cup and the second pier with a distance threshold; and determine the first stacking result based on the comparing results of the stacking groups.
15. 15 . The computer device of claim 14 , wherein the material-cage image comprises a first material-cage image and a second material-cage image; and the processor configured to obtain the material-cage image is configured to: obtain the first material-cage image by photographing the first stacking apparatus in a first direction of the first material cage and the second stacking apparatus in the first direction of the second material cage with the image obtaining device; and obtain the second material-cage image by photographing the first stacking apparatus in a second direction of the first material cage and the second stacking apparatus in the second direction of the second material cage with the image obtaining device, wherein the first direction is different from the second direction.
16. 16 . The computer device of claim 15 , wherein the first stacking apparatus and the second stacking apparatus each comprise foot cups and piers matched with the foot cups; the processor configured to obtain the first material-cage image is configured to: obtain the first material-cage image by photographing a foot cup in the first direction of the first material cage and a pier in the first direction of the second material cage with the image obtaining device; and the processor configured to obtain the second material-cage image is configured to: obtain the second material-cage image by photographing a foot cup in the second direction of the first material cage and a pier in the second direction of the second material cage with the image obtaining device.
17. 17 . The computer device of claim 14 , wherein the processor configured to perform the first target detection is configured to: obtain the feature information of the material-cage image with the first detection model, and recognize the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage according to the feature information.
18. 18 . A non-volatile computer-readable storage medium configured to store computer programs which, when executed by a computer, enable the computer to: obtain a material-cage image by photographing a first stacking apparatus of a first material cage and a second stacking apparatus of a second material cage; perform first target detection on the material-cage image with a first detection model to recognize the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage; determine first location information of the first stacking apparatus and second location information of the second stacking apparatus, and determine a first stacking result based on the first location information and the second location information; perform second target detection on the material-cage image with a second detection model to extract feature information of the material-cage image, and obtain a second stacking result based on feature information; and determine whether the first material cage is able to be stacked on the second material cage, based on the first stacking result and the second stacking result; wherein the first material cage has a first foot cup on a first surface of the first material cage and a first pier on a second surface of the first material cage, the second material cage has a second foot cup on a first surface of the second material cage and a second pier on a second surface of the second material cage, and wherein determining the first location information of the first stacking apparatus and the second location information of the second stacking apparatus, and determining the first stacking result based on the first location information and the second location information comprises: determining the first location information of the first foot cup in the first stacking apparatus and the second location information of the second pier in the second stacking apparatus; obtaining a distance between the first foot cup and the second pier in each of stacking groups based on the first location information and the second location information, wherein the first foot cup and the second pier in a same stacking group are aligned with each other in a vertical direction when the first material cage is stacked on the second material cage; for each of the stacking groups, obtaining a comparing result of the stacking group by comparing the distance between the first foot cup and the second pier with a distance threshold; and determining the first stacking result based on the comparing results of the stacking groups.
19. 19 . The method of claim 1 , wherein determining a final stacking result, wherein the final stacking result is to determine whether the first material cage is able to be stacked on the second material cage, based on the first stacking result and the second stacking result; when the final stacking result indicates that the first material cage is able to be stacked on the second material cage, a warehousing unmanned forklift triggers operations of stacking the first material cage on the second material cage to directly stack the first material cage on the second material cage; and when the final stacking result indicates that the first material cage is not able to be stacked on the second material cage, the warehousing unmanned forklift triggers operations of preventing the first material cage from being stacked on the second material cage, reports an error, and stops a current action.
20. 20 . The method of claim 1 , wherein the first detection model is Yolov5 model deployed through OpenVINO.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application is a continuation application of U.S. application Ser. No. 17/832,951, filed on Jun. 6, 2022, which claims priority under 35 U.S.C. § 119 (a) to Chinese Patent Application No. 202110648779.8, filed on Jun. 10, 2021, the contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD This disclosure relates to the technical field of warehousing and logistics, and in particular to a method for determining material-cage stacking, a computer device, and a storage medium. BACKGROUND With development of warehousing and logistics technology, goods are usually transported and stored with material cages during warehousing, and with material-cage stacking, it is possible to realize three-dimensional storage and reduce occupation for storage space. In traditional warehousing scenarios, a driver needs to extend his head out of a forklift to check a status of material cages whenever the driver drives the forklift to stack the material cages, so as to determine whether the material cages can be stacked. However, every time the material cages need to be stacked, the driver needs to manually determine whether stacking is possible, which reduces efficiency of material-cage stacking. SUMMARY Based on the above, it is necessary to provide a method for determining material-cage stacking, a computer device, and a storage medium for the above technical problems. A method for determining material-cage stacking is provided. The method includes the following. A material-cage image is obtained by photographing a first stacking apparatus of a first material cage and a second stacking apparatus of a second material cage. First target detection is performed on the material-cage image with a first detection model to recognize the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage. First location information of the first stacking apparatus and second location information of the second stacking apparatus are determined, and a first stacking result is determined based on the first location information and the second location information. Second target detection is performed on the material-cage image with a second detection model to extract feature information of the material-cage image, and a second stacking result is obtained based on the feature information. Whether the first material cage can be stacked on the second material cage is determined based on the first stacking result and the second stacking result. A computer device is provided. The computer device includes a processor and a memory. The memory is configured to store computer instructions which, when executed by the processor, enable the processor to execute the following. A material-cage image is obtained by photographing a first stacking apparatus of a first material cage and a second stacking apparatus of a second material cage with an image obtaining device. First target detection is performed on the material-cage image with a first detection model to recognize the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage. First location information of the first stacking apparatus and second location information of the second stacking apparatus are determined, and a first stacking result is determined based on the first location information and the second location information. Second target detection is performed on the material-cage image with a second detection model to extract feature information of the material-cage image, and a second stacking result is obtained based on the feature information. Whether the first material cage can be stacked on the second material cage is determined based on the first stacking result and the second stacking result. A non-volatile computer-readable storage medium is provided. The non-volatile computer-readable storage medium is configured to store computer programs which, when executed by a computer, enable the computer to execute the following. A material-cage image is obtained by photographing a first stacking apparatus of a first material cage and a second stacking apparatus of a second material cage. First target detection is performed on the material-cage image with a first detection model to recognize the first stacking apparatus of the first material cage and the second stacking apparatus of the second material cage. First location information of the first stacking apparatus and second location information of the second stacking apparatus are determined, and a first stacking result is determined based on the first location information and the second location information. Second target detection is performed on the material-cage image with a second detection model to extract feature information of the material-cage image, and a second stacking result is obtained based on feature information. Whether the first material cage can be