CN-121982356-A - Warehouse material arrival consistency verification method and system

CN121982356ACN 121982356 ACN121982356 ACN 121982356ACN-121982356-A

Abstract

The application provides a storage material arrival consistency verification method and a system, which belong to the technical field of intelligent storage, wherein a robot upper computer issues a arrival verification task to a WMS through an interface, and then receives a standard image, a nameplate field template and technical parameters returned by the WMS and caches the standard image, the nameplate field template and the technical parameters to the local; the robot collects images around the goods and materials, locates nameplate areas from the images, performs image preprocessing, extracts nameplate text information in an OCR mode, extracts material appearance feature vectors from the collected goods and materials images, extracts nameplate area feature vectors from the nameplate text information, calculates image similarity scores and text consistency scores, fuses the image similarity scores and the text consistency scores to generate comprehensive confidence, and generates a check conclusion according to comparison results of the comprehensive confidence scores and preset thresholds. The application realizes the automatic checking of the whole flow of the storage materials, shortens the check time of single piece, has low false recognition rate, synchronously updates the storage, and meets the storage tracing requirement.

Inventors

YAO HUIQUN
XU YANFA
WANG XIAOMIN
WANG ZONGGUANG
ZHAO HONGWEI
DENG HAO
QI LUFENG
LIU TAO
WANG HENG

Assignees

山东鲁软数字科技有限公司

Dates

Publication Date: 20260505
Application Date: 20251205

Claims (10)

1. The method for checking the consistency of the arrival of the storage materials is characterized by comprising the following steps: S1, a robot upper computer issues a cargo verification task to a warehouse management system WMS through an interface, and then receives a standard image, a nameplate field template and technical parameters corresponding to a current order returned by the warehouse management system WMS and caches the standard image, the nameplate field template and the technical parameters to a local place; S2, the robot collects images at a preset angle around the goods and materials, positions nameplate areas from the collected images, performs image preprocessing, and extracts nameplate text information from the preprocessed images in an OCR mode; S3, extracting a material appearance feature vector from the collected material image of the arrival goods by the upper computer of the robot, extracting a nameplate area feature vector from nameplate text information, calculating an image similarity score according to the material appearance feature vector and a standard appearance feature vector corresponding to the standard image, and calculating a text consistency score according to the nameplate area feature vector and a nameplate field template; s4, fusing the image similarity score and the text consistency score according to a preset weight to generate a comprehensive confidence coefficient, and generating a check conclusion according to a comparison result of the comprehensive confidence coefficient and a preset threshold value: If the comprehensive confidence coefficient is greater than or equal to a preset threshold value, judging that the verification is passed, uploading the receipt to a storage management system WMS, and updating the stock state; if the comprehensive confidence coefficient is lower than a preset threshold value, marking abnormality, triggering an acousto-optic prompt, waiting for manual review, and writing the review result back to the storage management system WMS to form a closed-loop record.
2. The method for verifying the consistency of warehouse materials to goods as in claim 1, wherein the step S1 comprises the following specific steps: s11, the upper computer of the robot issues a cargo checking task to a storage management system WMS through a RESTful API interface provided based on FastAPI; S12, the warehouse management system WMS responds to the arrival checking task and returns a standard commodity map URL, a nameplate field template and technical parameters corresponding to the current order to the upper computer of the robot; s13, the robot upper computer downloads a standard commodity diagram according to the URL of the standard commodity diagram, analyzes a nameplate field template and generates structured metadata containing a standard image cache identifier and a field matching rule; S14, the upper computer of the robot scans RFID labels or bar codes of the goods and materials to obtain the on-site stacking coordinates of the goods and materials in the warehouse; s15, the robot upper computer writes the parsed standard image, nameplate field templates, technical parameters, structural metadata and on-site stack coordinates into a local database for caching.
3. The method for verifying the consistence of warehouse materials to goods as in claim 2, wherein the step S2 comprises the following specific steps: S21, the robot moves according to a preset path planning algorithm, and acquires the whole image of goods and materials from at least four preset angles through a carried multi-view camera; s22, triggering a built-in YOLOv-lite network model serving as a nameplate detection model by the upper computer of the robot, processing the acquired integral image, and outputting the position coordinates and the detection confidence of the nameplate region; S23, judging whether the detection confidence coefficient is lower than a preset threshold; If yes, the robot upper computer controls the robot to automatically trigger the re-shooting flow, and the step S21 is returned; If not, go to step S24; s24, cutting out a nameplate area image from the whole image by the upper computer of the robot based on the position coordinates, and sequentially carrying out pretreatment of denoising, color equalization and resolution normalization; S25, the robot upper computer carries out character recognition on the preprocessed nameplate area image by adopting a PaddleOCR engine, and carries out field extraction and semantic analysis by combining a regular expression and a BiLSTM-CRF model to generate nameplate text information containing models, specifications and manufacturers.
4. The method for checking the consistency of warehouse materials to goods as claimed in claim 3, wherein the step S21 comprises the following steps: S211, carrying out robot path planning, mechanical arm pose calculation and camera parameter adaptation by a robot upper computer based on the cached structured metadata and the on-site buttress coordinates, generating a shooting track and transmitting the shooting track to a motion controller of the robot; S212, the robot drives the multi-camera to execute four-way five-point shooting according to the instruction of the motion controller, and self-adaptively adjusts the light supplementing intensity; S213, carrying out preprocessing operations of denoising, color equalization, resolution normalization and perspective distortion correction on an original image of the collected integral image of the arrival goods and materials by the upper computer of the robot to generate a standard diagram meeting the input requirement of a YOLOv-lite network model; The specific steps of step S22 are as follows: S221, inputting the preprocessed standard image into a built-in YOLOv-lite network model by a robot upper computer, forward reasoning the YOLOv-lite network model through a backbone network, a characteristic pyramid network and a detection head, and outputting a vector containing an original predicted value for each preset anchor point frame, wherein the vector comprises a boundary frame coordinate offset Objective score original value Probability distribution for each class; Wherein, the Representing the center point coordinate offset of the bounding box, Representing the wide offset of the bounding box, Representing a high offset of the bounding box; S222, decoding an original predicted value output by the YOLOv-lite network model, and converting the original predicted value into boundary frame coordinates and normalized objectivity scores in image pixel coordinates; Center point coordinates of the bounding box And width and height The decoding calculation formula of (2) is as follows: the calculation formula of the objectivity score is as follows: Wherein, the , , , , Is the original value of the direct prediction of the model, Is a Sigmoid activation function, mapping the input value to the (0, 1) interval; , Is the coordinates of the current grid relative to the top left corner of the feature map; , The objectivity score represents the probability of any nameplate in the prediction frame; s223, calculating final detection confidence of each bounding box for the nameplate category: Detection confidence = objectivity score x nameplate category probability; Filtering out invalid boundary boxes with detection confidence coefficient lower than a preset confidence coefficient threshold; s224, generating a boundary frame set for the filtered residual boundary frames, and executing a non-maximum suppression algorithm to eliminate repeated detection of the same nameplate target; the non-maximum suppression algorithm calculates the intersection ratio between every two bounding boxes in the bounding box set: Wherein, the And Representing two different bounding boxes; the boundary box with the highest confidence coefficient is reserved, and the boundary box with IoU values exceeding a preset threshold value is removed; s225, outputting the finally reserved boundary frame information with the highest confidence coefficient after non-maximum value inhibition, namely the minimum circumscribed rectangular coordinate of the nameplate area And a corresponding detection confidence.
5. The method for verifying the consistency of warehouse materials to goods as in claim 3, wherein the step S3 comprises the following specific steps: s31, extracting a material appearance feature vector from an overall image acquired on site by using a ResNet-SE network by using a robot upper computer, and extracting a nameplate area feature vector from a nameplate area image by using a Tiny-ViT network; s32, calculating cosine similarity between the material appearance feature vector and a pre-stored standard appearance feature vector corresponding to the cached standard image by the upper computer of the robot to obtain an image similarity score: Wherein, the For the image similarity score to be a score, For the appearance characteristic vector of the materials extracted on site, Pre-storing standard appearance characteristic vectors; S33, the robot upper computer carries out key field accurate matching and auxiliary field editing distance matching on the extracted nameplate text information and the cached nameplate field template, counts the number of missed objects and generates a field consistency code table; S34, calculating a text consistency score by the upper computer of the robot based on the field consistency code table: Wherein, the For a consistency score of a single field, For the text field extracted by the OCR, Corresponding fields in the nameplate field template; s35, taking the average value of the consistency scores of all the key fields as a final text consistency score.
6. The method for verifying the consistence of warehouse materials to goods as in claim 5, wherein the step S4 comprises the following steps: S41, receiving a preset threshold value for online adjustment judgment input by a user by the upper computer of the robot, and receiving a preset weight coefficient of an image similarity score and a text consistency score input by the user; S42, the robot upper computer carries out weighted fusion on the image similarity score and the text consistency score according to a preset weight coefficient, and a calculation formula is as follows: Wherein, the In order to integrate the confidence level, A preset weight coefficient for scoring the similarity of the images, A preset weight coefficient for scoring the text consistency, and ; S43, mapping the comprehensive confidence into a final conclusion of passing, to-be-checked and not passing three layers by the upper robot computer.
7. The method of claim 6, further comprising the steps of report generation and return after step S4: The robot upper computer generates a structured acceptance report based on the check conclusion, the image similarity score, the text consistency score, the position coordinates of the nameplate area, the field consistency code table and the counted number of misses, wherein the acceptance report comprises the arrival order number, the material code, the final conclusion, the conclusion generation time, the storage path of the whole image and the counted field miss information; And pushing the acceptance report to a corresponding interface of the storage management system WMS by the robot upper computer, and caching the acceptance report in a local database according to the arrival order number and the time stamp index.
8. The method for checking for consistence of warehouse materials to goods according to claim 7, further comprising the step of voice prompt: After the image similarity comparison is completed, the robot broadcasts a preliminary appearance judgment result based on the image similarity score in a voice mode; in the nameplate parameter comparison process, the robot broadcasts the matching state of the key field through voice based on the field consistency code table and the miss number; After the final verification conclusion is generated, the robot broadcasts corresponding instructions according to the conclusion category.
9. The warehouse substance arrival consistency verification method as claimed in claim 7, further comprising a data trace back management step of: The robot upper computer carries out association storage on the collected integral image, the nameplate area image, the extracted material appearance feature vector and the nameplate area feature vector, the calculated image similarity score and the text consistency score, the generated field consistency code table, the used preset weight coefficient, the generated check conclusion and the acceptance report; Responding to a user query request and rendering a visual report with difference highlighting through an integrated interface, and responding to a data penetration viewing request, a batch export request and an electronic signature archiving request of the visual report by a user.
10. A warehouse material arrival consistency verification system, comprising: the task management module is deployed on the robot upper computer and is used for issuing a cargo verification task to the storage management system WMS through an interface and receiving a standard image, a nameplate field template and technical parameters returned by the storage management system WMS; The image acquisition and processing module is used for controlling the robot to acquire images at a preset angle, locating nameplate areas from the acquired images, preprocessing the images, and extracting nameplate text information from the preprocessed images in an OCR mode; The multi-mode feature extraction and comparison module is used for extracting a material appearance feature vector from the collected material image, extracting a nameplate area feature vector from nameplate text information, calculating an image similarity score according to the material appearance feature vector and a standard appearance feature vector corresponding to the standard image, and calculating a text consistency score according to the nameplate area feature vector and a nameplate field template; the decision and report module is used for fusing the image similarity score and the text consistency score according to a preset weight to generate a comprehensive confidence coefficient, and generating a check conclusion according to a comparison result of the comprehensive confidence coefficient and a preset threshold value: If the comprehensive confidence coefficient is greater than or equal to a preset threshold value, judging that the verification is passed, uploading the receipt to a storage management system WMS, and updating the stock state; If the comprehensive confidence coefficient is lower than a preset threshold value, marking abnormality and triggering an acousto-optic prompt, waiting for manual review, and writing a review result back to a storage management system WMS to form a closed-loop record; And the communication module is used for carrying out data interaction with the WMS and uploading a check conclusion and an acceptance report.

Description

Warehouse material arrival consistency verification method and system Technical Field The application belongs to the technical field of intelligent storage, and particularly relates to a storage material arrival consistency verification method and system. Background Along with the evolution of a computer vision algorithm and the improvement of computing power of edge computing equipment, the automatic identification based on images gradually enters a warehouse logistics link, and the method aims to replace the traditional manual checking mode. However, the existing check-up method has significant drawbacks: At present, the main stream scheme still takes manual visual comparison as a main part and is assisted by simple bar code scanning or single-mode image recognition. The manual checking mode is highly dependent on experience and concentration of warehouse manager, is easy to cause missing error detection judgment due to visual fatigue in a long-time and large-batch operation environment, and has low checking and accepting efficiency, and the check and inspection time of a single material is as long as a few minutes. The system based on the single-mode image recognition can only perform coarse-grain comparison on the appearance of the material package, and can not simultaneously locate and analyze key parameter information (such as model, specification, manufacturer and the like) in the nameplate area, so that the discrimination capability of the system is seriously insufficient in scenes with similar appearance and different models. In addition, the existing recognition scheme mostly adopts an isolated visual or text recognition process, and is difficult to cope with complicated and changeable interference conditions such as site illumination, shooting angles, nameplate fouling and the like. In summary, the technical limitation of the existing check-up mode of the arrival goods makes the existing system have higher false recognition rate in the practical application scene of complex model specification and various arrival goods batches, and a large amount of manpower is still required to be input for secondary check-up, thus not only increasing the operation cost, but also causing the check-up link to become the bottleneck of storage digital management. On the other hand, the existing verification process and a Warehouse Management System (WMS) lack of real-time and closed-loop data interaction, verification results often depend on paper records or manual secondary input, and the problems of inconsistent data, difficult information tracing and the like are easily caused, so that the high standard requirements of modern intelligent warehouse on the operation efficiency, accuracy and data traceability are difficult to meet. Disclosure of Invention In a first aspect, an embodiment of the present application provides a method for checking consistency of warehouse materials to goods, including the following steps: S1, a robot upper computer issues a cargo verification task to a warehouse management system WMS through an interface, and then receives a standard image, a nameplate field template and technical parameters corresponding to a current order returned by the warehouse management system WMS and caches the standard image, the nameplate field template and the technical parameters to a local place; S2, the robot collects images at a preset angle around the goods and materials, positions nameplate areas from the collected images, performs image preprocessing, and extracts nameplate text information from the preprocessed images in an OCR mode; S3, extracting a material appearance feature vector from the collected material image of the arrival goods by the upper computer of the robot, extracting a nameplate area feature vector from nameplate text information, calculating an image similarity score according to the material appearance feature vector and a standard appearance feature vector corresponding to the standard image, and calculating a text consistency score according to the nameplate area feature vector and a nameplate field template; s4, fusing the image similarity score and the text consistency score according to a preset weight to generate a comprehensive confidence coefficient, and generating a check conclusion according to a comparison result of the comprehensive confidence coefficient and a preset threshold value: If the comprehensive confidence coefficient is greater than or equal to a preset threshold value, judging that the verification is passed, uploading the receipt to a storage management system WMS, and updating the stock state; if the comprehensive confidence coefficient is lower than a preset threshold value, marking abnormality, triggering an acousto-optic prompt, waiting for manual review, and writing the review result back to the storage management system WMS to form a closed-loop record. Further, the specific steps of step S1 are as follows: s11, the upper computer of the robot issue